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FOREWORD 



The proliferation of knowledge now makes it most difficult for scientists or 
engineers to keep ahead of change even in their own fields, let alone in contiguous 
fields. One of the fields where recent change has been most noticeable, and in fact 
exponential, has been automatic control. This three- volume Handbook will aid 
individuals in almost every branch of technology who must constantly refresh their 
memories or refurbish their knowledge about many aspects of their work. 

Automation, computation, and control, as we know them, have been evolving 
for centuries, but within the last generation their impact has been felt in nearly 
every segment of human endeavor. Feedback principles were exploited by Leonardo 
da Vinci and applied by James Watt. Some of the early theoretical work of im- 
portance was contributed by Lord Kelvin, who also, together with Charles Babbage, 
pointed the way to the development of today's giant computational aids. Since 
about the turn of the present century, the works of men like Minorsky, Nyquist, 
Wiener, Bush, Hazen, and Von Neuman gave quantum jumps to computation and 
control. But it was during and immediately following World War II that quantum 
jumps occurred in abundance. This was the period when theories of control, new 
concepts of computation, new areas of application, and a host of new devices ap- 
peared with great rapidity. Technologists now find these fields charged with chal- 
lenge, but at the same time hard to encompass. From the activities of World War II 
such terms as servomechanism, feedback control, digital and analog computer, 
transducer, and system engineering reached maturity. More recently the word 
automation has become deeply entrenched as meaning something about the field 
on which no two people agree. 

Philosophically minded technologists do not accept automation merely as a third 
Industrial Revolution. They see it, as they stand about where the editors of this 
Handbook stood when they projected this work, as a manifestation of one of the 
greatest Intellectual Revolutions in Thinking that has occurred for a long time. They 
see in automation the natural consequence of man's urge to exploit modern science 
on a wide front to perform useful tasks in, for example, manufacturing, transporta- 
tion, business, physical science, social science, medicine, the military, and govern- 
ment. They see that it has brought great change to our conventional way of think- 
ing about the human use of human beings, to quote Norbert Wiener, and in turn 
about how our engineers will be trained to solve tomorrow's engineering problems. 
They even see that it has precipitated some deep thinking on the part of our indus- 
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trial and union leadership about the organization of workers in order not to hold 
captive bodies of workmen for jobs that automation, computation, and control have 
swept or will soon sweep away. 

Perhaps the important new face on today ; s technological scene is the degree to 
which the broad field needs codification and unification in order that technologists 
can optimize their role to exploit it for the general good. One of the early instances 
of organized academic instruction in the field was at The Massachusetts Institute 
of Technology in the Electrical Engineering Department in September 1939, as a 
course entitled Theory and Application of Servomechanisms. I can well recollect 
discussions around 1940 with the late Dr. Donald P. Campbell and Dr. Harold L. 
Hazen, which led temporarily to renaming the course Dynamic Analysis of Auto- 
matic Control Systems because so few students knew what "servomechanisms" 
were. But when the GI's returned from war everybody knew, and everyone wanted 
instruction. Since that time engineering colleges throughout the land have elected 
to offer organized instruction in a multitude of topics ranging from the most ab- 
stract mathematical fundamentals to the most specific applications of hardware. 
Textbooks are available on every subject along this broad spectrum. But still the 
practicing control or computer technologist experiences great difficulty keeping 
abreast of what he needs to know. 

As organized instruction appeared in educational institutions, and as industrial 
activity increased, professional societies organized groups in the areas of control and 
computation to meet the needs of their members to tell one another about technical 
advances. Within the past five years several trade journals have undertaken to 
report regularly on developments in theory, components, and systems. The net 
effect of all this is that the technologist is overwhelmed with fragmentary, some- 
times contradictory, redundant information that comes at him at random and in 
many languages. The problem of assessing and codifying even a portion of this 
avalanche of knowledge is beyond the capabilities of even the most able technologist. 

The editors of the Handbook have rightly concluded that what each technologist 
needs for his long term professional growth is to have a body of knowledge that is 
negotiable at par in any one of a number of related fields for many years to come. It 
would be ideal, of course, if a college education could give a prospective technologist 
this kind of knowledge. It is in the hope of doing this that engineering curricula 
are becoming more broadly based in science and engineering science. But it is un- 
likely that even this kind of college training will be adequate to cope with the con- 
sequences of the rapid proliferation of technology as is manifest in the area of 
automation, computation, and control. Hence, handbooks are an essential com- 
ponent of the technical literature when they provide the unity and continuity that 
are requisite. 

I can think of- no better way to describe this Handbook than to say that the 
editors, in both their organization of material and selection of substance, have 
given technologists a unified work of lasting value. It truly represents today's 
optimum package of that body of knowledge that will be negotiable at par by 
technologists for many years to come in a wide range of disciplines. 

Gordon S. Brown 

Massachusetts Institute of Technology 




Accelerated advances in technology have brought a steady stream of 
automatic machines to our factories, offices, and homes. The earliest 
automation forms were concerned with doing work, followed by the con- 
trolling function, and recently the big surge in automation has been 
directed toward data handling functions. New devices ranging from 
digital computers to satellites have resulted from military and other 
government research and development programs. Such activity will con- 
tinue to have an important impact on automation progress. 

One of the pressures for the development of automation has been the 
growing complexity and speed of business and industrial operations. But 
automation in turn accelerates the tempo of whatever it touches, so that 
we can expect future systems to be even larger, faster, and more complex. 
While a segment of engineering will continue to mastermind, by rule of 
thumb procedures, the design and construction of automatic equipment 
and systems, a growing percentage of engineering effort will be devoted to 
activities that may be classified as problem solving. The activities of the 
problem solver involve analysis of previous behavior of systems and equip- 
ment, simulation of present situations, and predictions about the future. 
In the past, problem solving has largely been practiced by engineers and 
scientists, using slide rules and hand calculators, but with the advent of 
large-scale data processing systems, the range of applications has been 
broadened considerably to include economic, government, and social activi- 
ties. Air traffic control, traffic simulation, library searching, and language 
translation, are typical of the problems that have been attacked. 

This Handbook is directed toward the problem solvers — the engineers, 
scientists, technicians, managers, and others from all walks of life who are 
concerned with applying technology to the mushrooming developments in 
automatic equipment and systems. It is our purpose to gather together 
in one place the available theory and information on general mathematics, 
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feedback control, computers, data processing, and systems design. The 
emphasis has been on practical methods of applying theory, new techniques 
and components, and the ever broadening role of the electronic computer. 
Each chapter starts with definitions and descriptions aimed at providing 
perspective and moves on to more complicated theory, analysis, and appli- 
cations. In general, the Handbook assumes some engineering training and 
will serve as an information source and refresher for practicing engineers. 
For management, it will provide a frame of reference and background ma- 
terial for understanding modern techniques of importance to business and 
industry. To others engaged in various ramifications of automation sys- 
tems, the Handbook will provide a source of definitions and descriptive 
material about new areas of technology. 

It would be difficult for any one individual or small group of individuals 
to prepare a handbook of this type. A large number of contributors, each 
with a field of specialty, is required to provide the engineer with the desired 
coverage. With such a broad field, it is difficult to treat all material in a 
homogeneous manner. Topics in new fields are given in more detail than 
the older, established ones since there is a need for more background 
information on these new subjects. The organization of the material is in 
three volumes as shown on the inside cover of the Handbook. Volume 1 is 
on Control Fundamentals, Volume 2 is concerned with Computers and Data 
Processing, and Volume 3 with Systems and Components. 

In keeping with the purpose of this Handbook, Volume 1 has a strong 
treatment of general mathematics which includes chapters on subjects not 
ordinarily found in engineering handbooks. These include sets and rela- 
tions, Boolean algebra, probability, and statistics. Additional chapters are 
devoted to numerical analysis, operations research, and information theory. 
Finally, the present status of feedback control theory is summarized in 
eight chapters. Components have been placed with systems in Volume 3 
rather than with control theory in Volume 1, although any discussion of 
feedback control must, of necessity, be concerned with components. 

The importance of computing in research, development, production, real 
time process control, and business applications, has steadily increased. 
Hence, Volume 2 is devoted entirely to the design and use of analog and 
digital computers and data processors. In addition to covering the status 
of knowledge today in these fields, there are chapters on unusual computer 
systems, magnetic core and transistor circuits, and an advanced treatment 
of programming. Volume 3 emphasizes systems engineering. A part of the 
volume covers techniques used in important industrial applications by 
examining typical systems. The treatment of components is largely con- 
cerned with how to select components among the various alternates, their 
mathematical description and their integration into systems. There is also 
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a treatment of the design of components of considerable importance today. 
These include magnetic amplifiers, semiconductors, and gyroscopes. 

We consider this Handbook a pioneering effort in a field that is steadily 
pushing back frontiers. It is our hope that these volumes will not only 
provide basic information on new fields, but will also inspire work and 
further research and development in the fields of automatic control. The 
editors are pleased to acknowledge the advice and assistance of Professor 
Gordon S. Brown and Professor Jerome S. Wiesner of the Massachusetts 
Institute of Technology, and Dr. Brockway McMillan of the Bell Tele- 
phone Laboratories, in organizing the subject matter. To the contributors 
goes the major credit for providing clear, thorough treatments of their 
subjects. The editors are deeply indebted to the large number of specialists 
in the control field who have aided and encouraged this undertaking by 
reviewing manuscripts and making valuable suggestions. Many members 
of the technical staff and secretarial staff of The Ramo-Wooldridge Corpora- 
tion have been especially helpful in speeding the progress of the Handbook. 

Eugene M. Grabbe 
Simon Ramo 
Dean E. Wooldridge 
August 1958 
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1. SETS 

A set is a collection of objects of any sort. The words class, family, 
ensemble, aggregate are synonyms for the term set. 

Each object in a set is called an element (member) of the set. If S de- 
notes the set and b an element of S, one writes : 

bCS; 

this is read: "b belongs to S." If 6 does not belong to S, one writes: b (£ S. 

Sets will generally be designated by capital letters, elements by lower 
case letters. 

Important Examples of Sets. Z, the set of positive integers z; Z con- 
sists of the numbers 1, 2, 3, • • • ; 

J, the set of all integers j (including and the negative integers) ; 

1-01 
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Q, the set of rational numbers q (fractions a/b, where a is an integer and 
b is a positive integer) ; 

R, the set of all real numbers r (numbers which are expressible as unend- 
ing decimals) ; 

C, the set of all complex numbers c (numbers of form x + y V — 1, where 
x and y are real) . 

In geometry one employs sets of points; for example, all points on a speci- 
fied line or all points inside a circle. 
Geometric diagrams can be helpful in 
reasoning about sets which may have 
no reference to geometry (Fig. 1). 

A set can be designated by listing 
its elements between braces. Thus 

■S= {1, -3,7} 
Fig. 1. Set and subset. T is a subset 

of S, T c S. is the set whose elements are the num- 

bers 1, —3, and 7. For infinite sets 
one still uses braces, but instead of writing all the elements one gives a 
rule for set membership. For example, Z = \z \ z is a positive integer} is an 
abbreviation for "Z is the set of all z for which z is a positive integer." 
This set is also sometimes designated (less precisely) by 

Z= {1,2,3, ...,n, ••■}. 

Two sets are said to be equal if they have exactly the same members. 
For example, if 

A= {1,3,5,7}, B= {3,1,5,1,7}, 

then A = B. Neither the order in which the elements are written down 
nor the number of times that an element is repeated within the braces is 
significant. If A is not equal to B, one writes: A ^ B. 

Subsets. The set T is said to be a subset of the set S (Fig. 1) if every 
member of T is also a member of S; in symbols, 

T C S or SdT. 

If both T czS and S C T, S and T are equal. 

If S Z) T and S 9^ T, i.e., if S contains every element of T and at least 
one element not in T, one says that S contains T properly or that T is a 
proper subset of $; in symbols, 

S > T or T <S. 
The symbol 

T = { 1 1 / C S and t has properties P, Q } 
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indicates that T is the subset of S consisting of precisely those elements t 
of S which have the properties P and Q. 

Example. If S is a set of people, one might consider the subset T of S 
consisting of those members of S who are college graduates and are married : 

T = { t 1 1 G S and t is a college graduate and t is married } . 

Another example. If R is the set of all real numbers, then 

T = { 1 1 1 C R and 1 ^ t ^ 3 } is called a closed interval. 

The symbol is used to designate the empty (void) set, which has no ele- 
ments; is a subset of every set. 

The collection of all subsets of a given set S is called the power set of S 
and is denoted by 2 s . Thus 

2 s = {£7|Z7isasubsetofS}. 

The notation is suggested by the fact that if S is a finite set consisting of n 
elements, then 2 s has 2 n elements. 

Difference, Complement, Union and Intersection. If S and T are 

sets, the difference S — T is defined to be the set consisting of those ele- 
ments of S which are not in T, thus 

S-T = {s\s C S and s%_T). 

If T is a subset of S, then S — T is called the complement of T in $ and is 
denoted by C S T or even by CT if there can be no doubt about what $ is. 

The union of S and T, denoted by S U T, is the set consisting of those 
elements in S or in T (or in both). 

The set (S — T) U (T — S) is termed the symmetric difference of S and 
T and is commonly denoted by S © T. Thus 

S@T= {s|s££andsC T) U {s\s G T and s £ S). 

The intersection of S and T, denoted as S PI T, is the set consisting of 
those elements common to S and T. Thus 

Snr= {y\y CS and y€T}. 

Difference, symmetric difference, complement, union, and intersection 
are illustrated in Fig. 2. 

The union of a finite number of sets Si, S 2 , • • •, S n is defined as the set 
consisting of all elements belonging to at least one of the sets $i, • - • , S n . 
In symbols, 

n 

(J S a = {t\ there is at least one $ a for, which t C. S a }. 

a=l 
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If one denotes by A = { 1, 2, • • •, n} the set over which the index a varies, 
one can also denote this set by [J S a . The same notation can be used 

when A is an arbitrary index set, not necessarily finite. 



SUT 



S®T 




U = SHT 



-S 



:onn . 

Fig. 2. Set operations. 

The intersection of Si, S 2 , • • •, S n is denned as the set consisting of all 
elements common to all of Si, S 2 , • • •, S n . In symbols, 



fl^ = {t\t £S a for a = 1,2, - - ■ , n} 



a=l 



= n<s«. 

aCA 



If one restricts attention to subsets of one fixed set U, and complements 
are taken with respect to U, then there is a duality between union and 
intersection: 

COS U T) = CS fl CT, 

C(S n T) = CS U C7 7 . 

Accordingly, if all sets are replaced by their complements, all unions are 
replaced by intersections, all intersections by unions. 

Union, intersection, and complement satisfy several other laws, of which 
the following are typical : 



iU5 = 5Ui, 



a n (b u o = (a n b) u (a n c>. 



These come under the theory of Boolean algebra (Sec. 7 and Chap. 11). 

Cartesian Product. The Cartesian product S X T of two sets $ and 
T 7 is the set of all ordered pairs (s, t), where s CS and t £ 7\ Thus, if 

£= {a, 6}, T= {1,2,3}, 

then S X T consists of the 6 pairs 

(a, 1), (a, 2), (a, 3), (6, 1), (6,2), (6,3). 
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If R denotes the set of all real numbers, then R X R consists of all ordered 
pairs (x, y), where x and y are real; thus R X R can be represented by the 
points (x, y) of the zy-plane of analytic geometry. 

The Cartesian product of sets Si, ■ • ■ , S n in the order given is the set 

n 

of all ordered n-tuples (vectors) (Si, • • •, S n ), where Si C Si (i = 1, 2, 
• • • , n). If Si, ' • -, $ n , • • • is an infinite sequence of sets, one denotes the 
Cartesian product by 

00 

1=1 
this is the set of all infinite sequences (s{, • • ■, s n , • • •), where Si C Si 

(i = 1, • ••,/!, •••)• 

2. RELATIONS 

Let two sets A, B be given. A binary relation R is said to be given be- 
tween elements a of A and b of B if, for certain pairs (a, b), the relation R 
is valid, while for all others it is invalid. For example, let A = {1, 2, 3}, 
B = {4, 6, 8} and let the relation R between a and b be the condition that 
a equal l /$>. The relation is valid for the pairs 

(2, 4), (3, 6) 
and for no others. 

In general, specification of a relation R is equivalent to selection of the 
pairs (a, b) for which R is to be valid. This set of pairs is a subset of the 
Cartesian product A X B. Hence a relation R is equivalent to selection 
of a subset of A X B. This subset can also be denoted by R. 

If (a, b) is a pair for which the relation R is valid, one can write: aRb. 
In particular cases, other symbols can be used. In the example given 
above, one can write: a = }/<$>. 

When A = B, the relation R holds for certain pairs (a, b) of elements of 
A. One says: R is a binary relation in A. If A is the set of positive inte- 
gers, R can be chosen as the relation of "being less than"; in symbols, aRb 
becomes a <b. One can then interpret R as the set of all pairs (a, b) of 
positive integers for which a < b. Thus, instead of writing 2 < 3, one 
can write: (2, 3) £ R, to indicate that the relation is valid; similarly, (4, 3) 
(£ R, since 4 < 3 is false. 

Let R be a relation between elements a of A and b of B, so that aRb for 
certain pairs (a, b). Whenever aRb, one writes bR T a and calls R T the 
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transpose of the relation R. For example, the transpose of the relation < is 
the relation > ; the transpose of the relation "is the wife of" is the rela- 
tion "is the husband of." 

Range and Domain. Let R be a relation between elements a of A and 
b of B. For each a in A one denotes by R{a) the set of all b in B for which 
aRb; R(a) is called the image of a under R. For each subset S of A one 
denotes by .ft OS) the set of all b in B for which aRb for at least one a in S; 
.ft OS) is called the image of S under R. The image of A under R, the set 
R(A), is called the range of R. 

The counterimage of an element b under R is the set of all elements a for 
which aRb; this is the same as the set R T (b). The counterimage of a subset 
£/ of B is the set of all a in A for which aRb for at least one b in U; this 
is the same as the set R T (U). The set R T (B) is called the domain 
of R. 

A relation is sometimes called a correspondence between its domain and 
its range. 

Product of Relations. If R is a relation between elements a of A and b 
of B and $ is a relation between elements b of B and elements c of C, the 
product relation (or composition) .ft*S is defined as a relation between ele- 
ments a of A and c of C, as follows: aRSc whenever for some b in B one 
has aRb and &Sc. 

Example. The product can be illustrated by a communications net- 
work. Let A, B, C be three sets of stations, let aRb mean that a can com- 
municate with b and let bSc mean that 6 can communicate with c. Then 
aRSc means that there is a two-stage communication link from a through 
some intermediate station b to c. For products of three relations one has 
the associative law A(BC) = (AB)C. 

3. FUNCTIONS 

A relation F between elements a of A and b of B is said to be a function 

if, for every a in A, F(a) is either empty or contains just one element. If 

in addition F has domain A, one says that F is a function on A into B. In 

this case one identifies F(a) with its unique element b and writes: b = F{a) 

or 

F t 
a — » 6 

The terms mapping and transformation are synonyms for function. A 
function FonA into 5 can be defined directly as a correspondence which 
assigns to each element a in A a unique image b = F{a) in 5. The set B 
is called a codomain of i<\ The range F(A) is a subset of B. If F(A) = B, 
one says that F is a function on A onto B. 

If both F and F T are functions, one says that F is a one-to-one function. 
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The transpose F is then termed the inverse of F and is denoted by F~ x . A 
one-to-one function on A onto B is called a one-to-one correspondence between 
A and B. In this case one has 

(FF T )a = F T (F(a)) = a for all a in A; 

FF T is the identity function Ea on A onto A: Ea(o) . = a; similarly, i^V = 

In classical analysis, a function F is often denoted by F(x). The symbol 
F(x) then has two meanings: the value of the function for a particular x, 
and the function as a whole. Similarly, F(x, y) denotes a function of two 
variables and also the value of the function for given x, y. 

4. BINARY RELATIONS ON A SET 

A relation R on a set A is said to be 

Identical if R = Ea, i.e., if aRb is equivalent to a = b; 

Reflexive if R Z) Ea, i.e., if aRa for all a in A; 

Irreflexive if R PI Ea = 0, i.e., if aRa for no a in A ; 

Transitive if ijj 2 C it!, i.e., if aRb and bRc imply aRc; 

Symmetric if R = jft r , i.e., if a/26 implies &.Ra; 

Antisymmetric ii R C\ R T a Ea, i.e., if ai?6 and bRa imply a = 6; 

Asymmetric if R C\ R T = 0, i.e., if for no a, b is ai26 and bRa; 

Acyclic if J? n PI E A = for all n, i.e., if ai-Ra 2 , a 2 .Ra3, • • • , a n _iRa n im- 
ply ai * a n ; 

Complete if J? U R T = A X A, i.e., if for each pair (a, 6) either ai26 or 
bRa; 

Trichotomous ii R U R T \J E A = A X A and R Ci R T = 0, i.e., if for 
each pair (a, 6) exactly one of the relations aRb, bRa, a = b holds. 

Note that a relation is asymmetric if and only if it is antisymmetric and 
irreflexive. 

Examples. The parallel relation on lines in a plane is symmetric, irre- 
flexive but not transitive (unless a line is defined to be parallel to itself). 
The relation < on the real numbers is asymmetric, transitive, trichotomous, 
and acyclic; whereas the relation ^ is antisymmetric, reflexive, transitive, 
and complete. The relation "is at least as good as" is reflexive, transitive, 
but not antisymmetric if there are two objects judged equally good. 

5. EQUIVALENCE RELATIONS 

By a partition of a set A is meant a subdivision of A into subsets, no two 
of which have an element in common. By an equivalence relation R in A 
is meant a relation R in A which is reflexive, symmetric, and transitive. 
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Each partition of A determines an equivalence relation in A; aRb holds 
when a and b are in the same subset of the partition. Conversely, each 
equivalence relation determines a partition of A; the subsets of the parti- 
tion are the sets R(a), i.e., the sets of form {b\bRa}. The sets R(a) are 
called equivalence classes. 

Equivalence is the basis of classification; the equivalence classes contain 
elements which, although not identical, can be regarded as alike or inter- 
changeable for some purpose. Example. The sorting of nuts and bolts is 
based on the equivalence relation "has the same size and shape as." A 
property shared by all elements of each equivalence class is called an in- 
variant. More formally, let R be an equivalence relation on a set A. A 
function F on A is said to be an invariant relative to R if aRb implies F(a) 
= F(b). For example, if R is the relation of congruence on a set A of tri- 
angles, then the function F(a) =■ area of triangle a is an invariant. 

A set of invariants F, G, • • • relative to a relation R is said to be com- 
plete if 

F(a) = F(b), 0(a) = G(b), • • • together imply aRb. 

The language "a necessary and sufficient for IT is that P lf P 2 , • • • all hold" 
frequently states that Pi, P 2 , • • • are a complete set of invariants for an 
equivalence relation associated with K. Many of the theorems of elemen- 
tary geometry fall in this class. 

One sometimes is interested in choosing from each equivalence class a 
representative from which one or more invariants can be easily calculated. 
Such representatives are said to be in normal form or standard form. More 
technically, let R be an equivalence relation on a set A. A function which 
assigns to each equivalence class R(a) one of its members is called a canoni- 
cal form relative to R. Thus in matrix theory (Chap. 3) one has canonical 
forms for row equivalence, equivalence, congruence, orthogonal congruence, 
and similarity. It is customary to select a representative which displays a 
complete set of invariants. 

6. OPERATIONS 

A function F assigning to each ordered pair (a,'b), with a in A and b in 
B, an element c of set C is called a (binary) operation on A X B. If 
F(a, b) = c, one also writes aFb = c. If A = B, F is called an operation 
on A. If also C = A, F is called an interior operation; otherwise F is called 
an exterior operation. 

Examples. Addition and multiplication of numbers are interior opera- 
tions. The scalar product of two vectors is an exterior operation. 

Let F be an interior operation on A and let R be an equivalence relation 
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on A . One says that F has the substitution property relative to R, if aRa' 
and bRb' imply (aFb)R(a'Fb'). Example. Let A be the set of all integers; 
let aRb mean that a and b have the same parity (both even or both odd). 
Then aRa' and bRb' imply that (a + b)R(a' + 6')- Thus addition has the 
substitution property relative to R. 

An exterior operation is said to have the substitution property relative 
to R if aRa' and bRb' imply (aFb) = (a'Fb r ). 

7. ORDER RELATIONS 

A relation ^ on a set A is said to be a partial order if 

(i) a ±S a (reflexivity) , 

(ii) a ^ 6 and 6 ^ c imply a ^ c (transitivity), 

(iii) a ^ b and b ^ a imply a = b (antisymmetry). 
If a ^ & and a ^ 6, one writes : a < b. The relation < is then asymmetric : 

(iv) for no a, 6 is a < 6 and 6 < a; 
it is also transitive. If a ^ b and b < c, one writes: & ^ a and c > 6 
(transposition). An element a of A is said to be an upper bound for the 
subset B of A if b fS a for all b in 5; if also a ^ c f or every upper bound c 
of B, one says that a is a least upper bound (l.u.b.) for B. An upper bound 
for B which belongs to B is called a maximal element of B. 

If in these definitions one replaces ^ by ^, the resulting concepts are 
called lower bound, greatest lower bound (g.l.b.) and minimal element, respec- 
tively. 

The least upper bound (greatest lower bound) of a set, if it exists, is 
unique. 

The partial order is said to be a linear order or chain order, if it is com- 
plete: 

(v) for every a, b either a ^ b or 6 ^ a. 

Examples. The relation a ^ b between real numbers is a linear order; 
there is no maximal or minimal element. The complex numbers x -\- yi 
can be partially ordered by the definition: a -f- bi ^ c + di if a < c or if 
a = c and b = d. Numbers with the same real part are not compared. 

A partially ordered set A is said to be a lattice if each subset containing 
two elements has a least upper bound and a greatest lower bound. 

Example. Let A be the class of all subsets of a given set B and let 
S ^ T if S is a subset of T, i.e., if S c T. Then A is a lattice and l.u.b. 
{S, T) = S U T, g.l.b. {S, T) = S PI T. One extends this notation to 
lattices generally and uses a U b (a cup 6) for l.u.b. {a,b}, a D b (a cap b) 
for g.l.b. {a, b}. If every subset of A has a g.l.b. and a l.u.b., then A is 
called a complete lattice. The example given of a class of all subsets of a 
given set is a complete lattice. 



1-10 GENERAL MATHEMATICS 

In a lattice the operations U and f] have the following properties: 
For all a, b, c in A 

a U b = b U a and a f) b = b D a (commutative laws) ; 
(a U b) U c = a U (b U c) and (a fl 6) fl c = a D (b fl c) (associa- 
tive laws) ; 
a I) a = a, a C\ a = a (idempotent laws) ; 
ofl (a U b) = a, a U (aTl b) = a (absorptive laws). 

A lattice is said to be distributive if for all a, b, cm. A 
a U (6 (1 c) = (a U b) fl (a U c) 

or, equivalently, if a fl (b U c) = (a PI b) U (a fl c) for all a, 6, c. 

If A has a minimal element and a maximal element, one ordinarily de- 
notes them by 0, 1 respectively. Two elements a and b are said to be com- 
plements of each other if a D b == and a U 6 = 1. A lattice is said to be 
complemented if each of its elements has a complement. In a distributive 
lattice, no element can have more than one complement. A Boolean alge- 
bra is a complemented, distributive lattice. (See Chap. 11.) 

Example. The partially ordered set formed of the class of all subsets 
of a given set B forms a Boolean algebra. The minimal and maximal ele- 
ments are the empty set 0, and the set B the complement is the same as 
that defined in Sect. 1. 

Remark. A relation <, which satisfies only conditions (i) and (ii) is 
called a preorder or quasi-order. An example is the relation "is at least as 
good as" between automobiles. 

8. SETS OF POINTS 

Sets of real numbers can be interpreted as sets of points on the real line, 
or number axis. For fixed a,b, a < b, 

{x\a < x < b} is an open interval, 
[x\a ^ x ^ b) is a closed interval, 
{x\a ^ x < b} or {x\a < x ^ 6} is a half-open interval. 

For fixed a, e, e > 0, the set 

{x\a — e < x < a -\- e} 

is the e-neighborhood of a. An arbitrary set of real numbers is open if each 
element of the set has an e-neighborhood contained in the set. A set is 
closed if its complement is open. A number a is a limit point of a set A if 
every e-neighborhood of a contains at least one element of A differing from 
a. 
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Sets of ordered number pairs (x, y) can be interpreted as sets of points 
in the zy-plane. For fixed (a, b) and e > the set 

{( X) y)\( x -a) 2 +(y-b) 2 <e} 

is the e-neighborhood of (a, b). A set of points in the xy-p\ane is open if 
each point (a, b) in the set has an e-neighborhood contained in the set. A 
set is closed if its complement is open. A point (a, b) is a limit point of a 
set A if every e-neighborhood of (a, b) contains at least one element of A 
differing from (a, b). An open set is called an open region or domain if each 
two points of the set can be joined by a broken line within the set. A 
point (a, b) is a boundary point of set A if every e-neighborhood of (a, b) 
contains at least one point of A and at least one point not in A. The 
boundary of A is the set of all boundary points of A. A closed region is a 
set formed of the union of an open region and its boundary. A point (a, b) 
of a set A is called an isolated point of A if some e-neighborhood of (a, b) 
contains no element of A other than (a, 6). 
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1. POLYNOMIALS 

A polynomial may be defined as a function / = f{x) defined by an equa- 
tion 

Six) = a n x n + a n _ x x n ~ x ^ h a x x + a Q , 

where the coefficients a , a\, • • • , a n are constants (real or complex) and x 
is variable (real or complex). The leading coefficient a n will be assumed 
7^ 0. The degree of / is n. A polynomial a 2 x 2 + aix + a of degree 2 is 
quadratic; a polynomial a^x + a , of degree 1 is linear; we accept the con- 
stant polynomials : f(x) = a®, although the zero polynomial f(x) = must 
be tacitly excluded from certain contexts. 

An algebraic equation of degree n is an equation of form : polynomial of 
degree n in x = 0; that is, of form 

f{x) = a n x n -\ h a = (a* 9*0). 

A root of such an equation is a value of x which satisfies it; a root of the 
equation is called a root of /(a 1 ) or a zero of Six) . Thus r is a root of Six) if 
and only if /(r) = 0. 
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The fundamental theorem of algebra asserts that an algebraic equation of 
degree n (n = 1, 2, • • •) has at least one root (real or complex) (Refs. 1, 2). 
From this it follows that an algebraic equation of degree n has exactly n 
roots (some of which may be repeated, see below). 

The operations of addition, subtraction, multiplication, and division of 
polynomials will be assumed to be familiar. 

Synthetic division is an abbreviation of division by a linear polynomial, 
x — c. As an illustration, the division of 3a; 2 — 7x + 11 by x — 2 is car- 
ried out in long form and by synthetic division. 





Sx - 1 


X 


- 2 | 3x 2 - 7x + 11 




Sx 2 - 6x 




- x + 11 




x + 2 



3-7 11 
6-2 



1 



9 



9 



Either method yields the quotient Sx — 1 and remainder 9, so that 

Sx 2 - 7x + 11 = (3x - l)(x - 2) + 9. 

In the synthetic process, on the first line one replaces x — 2 by +2, 
3x 2 — 7x -\- 11 by the numbers 3, —7, 11. A zero is placed below the 3 
and added to yield 3; the result is multiplied by 2 to yield 6; the 6 is added 
to —7 to yield —1; the —1 is multiplied by 2 to yield —2; the —2 is 
added to 11 to yield 9. On the third line the coefficients of the quotient, 
3.x — 1, and the remainder, 9, appear in order. 

Remainder Theorem. // a polynomial f(x) is divided by x — c, then the 
remainder is f(c) . 

Factor Theorem, c is a root of fix) if and only if x — c is a factor of 
f{x) (Ref. 2). 

Application. If one root, c, of fix) has been found, the remaining roots 
of fix) will be roots of the quotient polynomial fix) -f- (x — c), which is of 
degree n — 1. Repetition of this reasoning leads to a representation of /(x) 
as a constant times a product of linear factors (x — c{), (x — c 2 ), 
Since f(x) is of degree n there must be exactly n such factors: 

fix) = a n ix — Ci)ix — c 2 ) • • • ix — c n ). 

Thus fix) has n roots c x , c 2 , • • • , c w , some of which may be equal. If ci is 
repeated m times, so that ix — c\) m is a factor of fix) (and ix — c x ) m+1 is 
not a factor), then c\ is a root of multiplicity m. 
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Repeated Roots. If c is a repeated root of f(x) (a root of multiplicity 
2 or more), then c will also be a root of f'(x), the derivative of f(x), 

f'(x) = na n x n ~ l + (n - \)a n ^x n ~ 2 + • • • + a x . ' 

To find the repeated roots, one can proceed as follows. Let 

fo(x)=f(x), fi(x)=f'(x), 

and by division obtain 

/oO) = g\(x)fi(x) + /2O), 

where f 2 (x) is of degree lower than that of/i(x). Continue, taking 

ft-i(x) = g t (x)f t (x) +f t+ i(x), 

until ft+i(x) = 0. Then the repeated roots of f(x) are the roots of f t (x). 
If f t (x) is a (non-zero) constant, f(x) has no repeated roots. Otherwise all 
repeated roots of f(x) can be found as the roots of f t (x), which has degree 
lower than that of/ (Ref. 2). 

2. REAL ROOTS 

In this section f(x) denotes a polynomial with real coefficients. If f(x) 
is of odd degree, f(x) has at least one real root, whereas x 2 + 1, for exam- 
ple, has no real roots. Two problems will be considered: (1) establishing 
existence of real roots, perhaps within prescribed intervals; (2) computing 
to a satisfactory accuracy the value of a root that has been approximately 
located. 

Graphical Methods. One plots the graph of y = f{x). The roots of 
odd multiplicity are the values of x at which the curve crosses the a;-axis, 
while at roots of even multiplicity the curve is tangent to the x-axis. If 
f(x\) and f(x 2 ) have opposite signs, there is a root between x\ and x 2 . In 
practice, one could use synthetic division to compute the values of f(x) for 
a number of values of x within some interval a ^ x ^ 6. The values a and 
b can be chosen so that all roots lie between a and b; in particular, all real 
roots lie in the interval 

M 
+ l)^z^-— -+1, 



-(A +i ) 

\\a n \ / 



where M is the largest of the numbers \a \, \ai\, • • • , | a n -i | . Narrower 
bounds can often be found by inspection. If in computation of /(&) by 
synthetic division, the third row consists of non-negative numbers, then no 
real root exceeds b. An alternative criterion is Newton's rule: if the values 
f(b),f'(b), •••, / (n) (fe) of the successive derivatives are all non-negative, 
then no root exceeds b. These last two rules can be applied to the equa- 
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tion obtained by replacing x by —x, in order to obtain a lower bound a. 
The following rule is sometimes useful: if g(x) = x n f(l/x) and if g has all 
of its real roots between — b and +&, then / has no real roots between 
-(1/6) and (1/6). 

Derivative. The value f'(c) of the derivative at c gives the rate of 
increase (decrease, if /' (c) < 0) of f(x) at x = c. At an extremum (relative 
maximum or minimum) of f(x), fix) is zero; there can be at most n — 1 
such values of x {critical points of /(#)). 

Rolle's Theorem. Between each two real roots of f(x) there is at least one 
critical point. 

Descartes and Sturm Tests. Zero is a root of f(x) only if a = 0. By 
division by x or some power of x, all zero roots can be removed. Informa- 
tion about the number of positive roots is given by : 

Descartes'sRule. The signs of the coefficients a n , a n -i, • • •, a in order, 
omitting possible zeros, form a string of +'s and — 's. The number v of 
alternations in sign is defined as the number of consecutive pairs -\ — or 
— \- . The number p of positive roots is no greater than v and v — p is even. 
(Negative roots of f(x) are the positive roots of f(—x).) 

Examples, x 2 + x + 1 = 0, v = 0, no positive roots; re 2 — 2x + 3 = 0, 
v = 2, or 2 positive roots; x 2 + 2x — 3 = 0, v = 1, 1 positive root. A 
more precise criterion is given by: 

Sturm's Theorem. Write f (x) = f(x), fi(x) = f'(x) and, stepwise, 
ft-i(x) = qt(x)f t (x) — f t+ i(x), where ft+i(x) is of lower degree than f t (x). 
Continue until some f m +i(x) = 0. Now suppose a < b, f(a) ^ 0, fib) t^ 0. 
Let via) be the number of alternations in sign in the sequence of values f\ic), 
/2(c), • • •, fm(c) (zeros omitted). Then v(a) — v{b) is the exact number of 
distinct real roots between a and b (Ref. 2). 

Newton's Method. If X\ is an approximate value of a root of f(x) then 
one sets 

x 2 = xi - [fM/f'M], 

x 3 = x 2 - [f(x 2 )/f'(x 2 )], ••-. 

The sequence of numbers thus defined converges to a real root of f(x), 
provided f(xi)f"{x\) > and it is known that x± lies in an interval con- 
taining a root of fix) but none of f'(x) or of f"(x) (Ref. 2). 

3. COMPLEX ROOTS 

Let f(z) = a n z n + • ■ • + a be a polynomial in the complex variable z, 
z = x + iy,i = V — 1. The coefficients are allowed to be real or complex. 
If they are real, complex roots of fiz) come in conjugate pairs, x ± yi, so 
that the total number of nonreal complex roots is even. 
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If f(z) is of degree 2, 3, or 4, explicit algebraic formulas for all roots are 
available (Ref. 1). 

It is proved in Galois theory that similar formulas for equations of 
higher degree do not exist (Ref. 1). 

Equations for Real and Imaginary Parts. Replacement of z by 
x + iy in the equation f(z) = and equating real and imaginary parts 
separately to zero leads to two simultaneous equations in the real variables 
x, y. These can be solved by elimination. 

Example, z 3 — z -f- 1 = 0. Replacement of z by x + iy leads to the 
equations x 3 — 3xy 2 — x + 1 = 0, 3x 2 y — y 3 — y = 0. To find nonreal 
roots, one assumes y 5^ and is led to the equations 8x 3 — 2x — 1 = 0, 
if = 3x 2 — 1. The first has one real root x = 0.66. Hence 0.66 d= 0.55* 
are the nonreal roots of the equation. 

Application of Argument Principle. The argument principle, when 
applied to the polynomial/^), states that the total change in the argument 
(polar angle) of the complex number w = f(z), as z traces out a simple 
closed path (circuit) C, equals 2w times the number of zeros of f(z) inside 
C (provided f(z) ^ on C). (See Chap. 7, Sect. 5.) The path C can be 
chosen as a circle, semicircle, square, or other convenient shape, and the 
variation of the argument of w can be evaluated graphically. One can pass 
to the limit from a semicircle in order to find the number of roots in a half- 
plane. This is the basis of the Nyquist criterion (Chap. 21). 

In general, no root can lie outside the circle with center at z = and 
radius 1 + (M/\a n \), where M is the largest of |a |, |ai|, •••, |a w _i| 
(Ref. 2). 

Hurwitz-Routh Criterion. This is a rule for determining whether all 
roots of f(z) lie in the left half-plane (i.e., have negative real parts). For 
a given sequence c , C\, • • • , c n , • • •, one denotes by A& the determinant 



Afc = 



ci c 0-0 

C 3 C2 Ci C • 



C2k-1 C 2 k-2 



Ck 



so that Ai = Ci, 



Ao = 



Ci c 
c 3 c 2 



A* = 



c\ c 

c 3 C 2 
C 5 C 4 





c\ 
c 3 



For a given polynomial f(z) = c z n + c x z n x -\ \- c n with real coeffi- 
cients and Co > 0, one forms Ai, • • •, A n _i, with Cfc replaced by for 
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k > n. All roots of /(z) lie in the left half-plane if and only if Ai > 0, 
A 2 > 0, ••-, A n _! > (Ref. 3). 

Graffe's Method. Graffe's method is efficient for finding a complex root, 
or successively all roots, of a polynomial /(z). For simplicity, suppose that 
/(z) has no repeated roots, as can always be arranged by the methods indi- 
cated above (Sect. 1). One must further suppose that /(z) has a single root 
r of maximum absolute value ; if this fails for /(z) it will hold for the new 
polynomial g(z) = f(z + c) for all but certain special values of c. It is 
necessary to have some rough idea of the argument of the root r ; for 
example, if r is real, to know whether it is positive or negative. 

Starting with the polynomial f(z) = /i(z) = z n + a\z n ~ x -\ , one 

forms /i(— z). The product /i(z)/i(— z) contains only even powers of z, 
hence is of the form /i(z)/i(— z) = f 2 {z 2 ). Similarly, / 3 (z) is formed from 
/ 2 (z): fs(z 2 ) = f2(z)-f 2 (— z), and the process is continued to form a se- 
quence of polynomials fk(z) = z n -f- a^z n ~ x -f- • • • . (As justification note 
that fk has roots which are the 2 fc th powers of the roots of /; that — a k is 
the sum of the roots of fk, and hence that the ratio of — a^ to r 2fc approaches 
1 as k — > oo). One chooses a value z/ c of the 2 fc th root of —a^, the choice 
of z fc is made to agree as closely as possible in argument with the initial 
estimate for the argument of r . The successive values Z\, z 2 , • • ■ can be 
expected to approach r rapidly. 

After the root of largest absolute value has been found, one could divide 
out the corresponding factor and proceed to find the root of next largest 
absolute value. In practice, it is generally more efficient to use an elabora- 
tion of Graffe's method (Ref. 7). 
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1. VECTOR SPACES 

Let F denote the rational number system, or the real number system, or 
the complex number system; in the following, elements of F are termed 
scalars and are denoted by small Roman letters a, b, c, • • • . A vector space 
V over F is defined (Ref . 9) as a set of elements called vectors, denoted by 
small Greek letters a, /3, 7, • • • , for which the operations of addition : a + /3 
and multiplication by scalars : aa are defined and satisfy the following rules : 

(i) For each pair a, /3 in V, a + is an element of V and a + /3 = /3 + a, 

<* + 03 + 7) = (« + 18) + 71 
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(ii) For each a in V and each a in F, aa is an element of V and, for 
arbitrary b in F and /3 in V 

a(a + |8) = aa + a/3, (a + 6)a = aa + &a, 

a(6a) = (ab)a, la = a; 

(iii) For given a, jS in V, there is a unique vector 7 in F such that 
a + 7 = /3. In particular, there is a unique vector denoted by such that 
a + = a for all a in V. 

When F is the real number system, V is called a real vector space ; when 
F is the complex number system, V is a complex vector space. The system 
F can be chosen more generally as a field (Ref. 9). The vectors of me- 
chanics in 3-dimensional space form a real vector space V. In terms of a 
coordinate system, the elements of V are ordered triples (x, y, z) of real 
numbers; addition and multiplication by real scalars are defined as follows: 

Oi, yi, 21) + (x 2 , V2, 22) = Oi + x 2 , yi + 2/2, zi + 22), 

a(#, 2/, z) = (ax, ay, az). 

A vector a is said to be a linear combination of vectors ai, • • •, a n if 

a = a\a\ + • • • + a n a n 

for appropriate choice of a\, • • • , a n . An ordered set \a\, • • • , a n ) is said 
to be independent if no member of the set is a linear combination of the 
others or, equivalently, if 

a\ai ^ 1- a n (Xn = 

implies a\ — 0, • • •, a n = 0. If the ordered set S = {ai, • • •, a n } is inde- 
pendent and a is a linear combination of its elements (is linearly dependent 
on S), then the scalars a\, • • • , a n can be chosen in only one way so that 
a = ^ia^ai. 

If there is a finite set S = {ai, • • • , a n ) such that every a in V is linearly 
dependent on S, then V is said to be of finite dimension. For the remainder 
of this chapter, only vector spaces of finite dimension will be considered; this is, 
however, not the only case of importance. If S = {ai, • • •, a n ) is inde- 
pendent and every a of V is a linear combination of these vectors, then S 
is said to constitute a basis for V. Every finite dimensional vector space 
has at least one basis, all bases have the same number, n, of elements; n is 
the dimension of V. 

A subset W of V is said to be a subspace of V if, with the operations as 
defined in V, W is itself a vector space. A subset W will be a subspace of 
V if, whenever a, (3 are in W, a + /3 is in W, and aa is in W for every a in 
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F. In particular, {0} is a subspace, as is 7 itself. The intersection (Chap. 

1, Sect. 1) W fl U of two subspaces of 7 is a subspace of 7; it is the largest 
subspace contained in both W and U. The union W U U is not usually a 
subspace ; the smallest subspace containing W and U is rather their (linear) 
sum W -f- U, consisting of all vectors a + /3, a in 17, and /3 in i7. If 
IT (1 £7 = 0, then W + t/ is called a direct sum, and is often denoted by 
W ® U or W -f- U; in this case every vector in W + [/ is expressible 
uniquely as a + P, « in IF, /3 in £7. For any set of vectors {«i, •••,«„}, 
the set of all their linear combinations constitutes a subspace, and the sub- 
space is spanned by them. Every independent set is a subset of a basis. 
From this it follows that, for each subspace W, there exists U (in general, 
many) such that 7 is the direct sum of W and U. 

2. LINEAR TRANSFORMATIONS 

Let / be a transformation (function, mapping) (Chap. 1, Sect. 3) of 
vector space 7 into a second space V; f is said to be linear if for all a, /3, 
a, b, 

f(aa + 6/3) = af(a) + &/(/3). 

The image of V under/, denoted by f(V), is the set of all vectors f(a) for 
a in V; f(V) is a subspace of 7'. If /(F) = V, f is said to map V onto V. 
The null space of/, denoted by N(f), is the set of all vectors a in V such 
that /(a) = 0; N(f) is a subspace of V. If JV(/) contains only the ele- 
ment 0, / is said to be nonsingular; this is equivalent to the condition that 
/be one-to-one (Chap. 1, Sect. 3); a nonsingular transformation is termed 
an isomorphism of V onto f(V). The rank of / is defined as the dimension 
of f(V) ; this equals the dimension of V minus that of N(f). The mapping 
/ is nonsingular if and only if its rank is maximal, that is, equals the di- 
mension of V. If W is chosen so that V is the direct sum of N(f) and W, 
and W has dimension greater than 0, then the restriction of / to IF is a 
nonsingular mapping of W onto f(V), that is, an isomorphism of W onto 
f(V). If /is an isomorphism of V onto V , then the inverse transformation 
f T = / _1 is a linear transformation of V onto V. 

The set of all linear transformations of V into V becomes itself a vector 
space over F, if addition and multiplication by scalars are defined by the 
rules: . 

/ + g is the transformation such that (/ + g)a = f(a) + g(a) for all a 
in 7; 

af is the transformation such that (a/) a = a[f(a)] for all a in 7. 

If / maps 7 into V and g maps 7' into 7", following fbyg defines the 
composite transformation fg of 7 into 7"; explicitly, /gr (a) = g[f(a)]. Iff, 
g are linear, so also is fg (Refs. 2, 8, 9). 
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3. COORDINATES 

Let {«!,-••, a n } be a basis for the vector space V, so that every vector 
a in V can be written uniquely in the form 2a z aj. The a t - are the coordi- 
nates of a relative to the chosen basis; the a; are also termed components, 
but this word is sometimes used for the terms a^. The choice of a definite 
basis is often necessary for computation. With a fixed basis understood, 
one can replace each vector a by the corresponding w-tuple (oi, • • •, a n ); 
then 

(Ol, ••', Cln) + (&1, ••-,&»») = (Ol + &1, * • *, On + O n ), 

c(«i, • • •, a„) = (cai, • • •, ca n ). 

A basis that is natural at one stage of a problem may not be the most 
advantageous at a later stage, so that one must be prepared to change 
bases. 

If a basis a\, • ■ • , a n is chosen for V and a basis a\, • • •, a' m for V, 
then each linear transformation of V into V can be assigned coordinates 
as follows. The transformation / is fully determined by the images /(«»•) 
of the basis elements for V. If /(a;) = Zjdija'j, then /may be characterized 
by the n-m scalars a^-, where i = 1, • • -,n,j = 1, • • •, m. These numbers 
are usually thought of as arranged in a rectangular array, or matrix 



A = 



On 


Ol2 • 


a \m 


a 2 i 


022 


02 TO 


Onl 


O n 2 


o nm 



= (Off). 



One terms A the matrix representing the transformation / relative to the 
given bases in V and V. 

If g is a second transformation from V into V, with matrix B = (b;y), 
it is clear that the transformation / + g will have the matrix (a^- + &#). 
Accordingly, one defines the sww of two n by m matrices as follows: 

(ay) + (by) = (ay + by). 



Similarly, the product cA, which represents cf, is defined as the matrix 
(can). 

Now let / be a linear transformation of V into V, g a linear transforma- 
tion of V into V", where V, V have bases as before, and V" has a basis 
a" 1, • • •, a" p . Relative to these bases, / is represented by an n by m 
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matrix A = (a,-y), <j by an m by p matrix B = (&,-_,•), /<7 by an n by p matrix 
C = (en). Since 

(J(j)(<xi) = X Ha ij b jk a" k 
j k 
one finds 

m 
i=l 

correspondingly, one defines the product of two matrices A and B (where 
the number of columns of A equals the number of rows of B) to be the 
matrix C = AB, where the elements c^ of C are given by the above "row- 
by-column" rule. Multiplication of matrices is not commutative, but is 
associative and distributive: A(BC) = (AB)C, A(B + C) = AB + AC, 
(A + B)C = AC + BC. 

If a = Zaj-aj- is a vector with coordinate representation {a\, • • •, a n ), one 
can regard the n-tuple as a 1 by n matrix. The product aA can then be 
evaluated as that of a 1 by n matrix and an n by m matrix. The result is 
the 1 by m matrix 

(n n \ 

S a i a il> • ' ' , 23 a i a im J 

which represents /(a) : 

/(«) = f (J2 a i a i) = Z) a<f(ai) = SZ aidija'j 

\ i / i i j 

m / n \ 

= X ( X) CW ) a'y 

This shows that, when bases are chosen in V and V , each matrix A is the 
matrix of a linear transformation (Ref . 2, 9) . 

4. ECHELON FORM 

The matrix A associated with a linear transformation / from V to V 
can be given an especially simple form by suitable choice of basis for V, 
for V, or for both. We consider the effect of a change of basis for V. 
Every change of basis for V can be effected by a sequence of elementary 
transformations of the following types: (1) replacement of at by a scalar 
multiple ccti, c ^ 0; (2) renumbering, interchanging a { and «_,-; (3) adding to 
cti some multiple of ay, j ?* i, so that a; is replaced by a; + cctj (and ay is 
left unchanged). The effect of each transformation is to carry out the 
analogous operation on the rows of the matrix A = (a#). Thus (1) multi- 
plies each element of the ith. row by c, (2) interchanges ith and jth rows, 
(3) replaces the ith row by (an -f- caj\, • • • , ai m + cay OT ). 
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A matrix is said to be in (strict) echelon form (Ref. 9) if: 

(i) The leading element (first nonzero element) in each nonzero row ap- 
pears farther to the right than that of any preceding row; 

(ii) The leading elements are all 1; 

(iii) Only zeros appear in the same column with a leading element; 

(iv) All zero rows (if any) appear at the bottom. 
By a zero row (or column) is meant one consisting wholly of zeros. 

Example. The following matrix is in echelon form. 



A = 



13 5 

10 7 

12 

L0 0. 



Each matrix can be reduced to echelon form by elementary transforma- 
tions on its rows, as follows : 

Step 1. If the first column is a zero column, leave it untouched and 
proceed to the matrix formed by the remaining columns. If the first 
column is not a zero column, permute rows so that a n ^ 0. Dividing 
this row by a n gives a new matrix with a n = 1- Subtracting suitable 
multiples of this row from the other rows makes all an = for i 9^ 1. 
The matrix now has a first column which is a zero column, or else it has 
all zeros except for a 1 in the top position. Leave the first row and column 
untouched and proceed to the matrix formed by the elements not in the 
first row or column. 

Step 2. Repeat this process as long as possible. The resulting matrix 
will satisfy (i), (ii), and (iv). 

Step 3. To obtain (iii), subtract suitable multiples of each row from 
earlier rows to convert the elements in these rows above the leading ele- 
ment of the given row into zeros. The result may be stated as follows: 

Every matrix is row-equivalent to an echelon matrix and (it can be shown) 
to a unique echelon matrix. 

Application to Systems of Equations (Ref. 9). A system of m 
linear equations in n unknowns 



XI a-ijXj = Ci (i = 1, • • •, m) 

3=1 



can be replaced by a single matrix equation 

AX = C, 
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X = 



Xi 



-X n . 



C = 



C\ 



Let B be the augmented matrix of the system, obtained by adjoining — C 
as (n + l)st column to A. The usual manipulations of equations employed 
to successively eliminate (so far as possible) the unknowns x\, x 2 , • • •, x n 
correspond to elementary transformations on the matrix B. If the result 
were the echelon matrix of the above example, one would have obtained 
the equivalent system: 



x 2 + 3x 3 



x± 



+ 5 = 
+ 7 = 



" x s + 2 = 

Since xi, x s do not appear in leading terms, they can be assigned arbitrary 
values; the general solution can be obtained immediately from the given 
equations : 



x\ arbit., x 2 = —3x 3 — 5, x 3 arbit., 



x 4 



■7, 



x 5 



-2. 



If a row (00 • • • 01) had appeared, there would be an equation 1 = 0, as a 
consequence of the original system, which would therefore be inconsistent 
and have no solution. 

5. RANK, INVERSES 

The rank of a linear transformation / of V into V was defined (Sect. 2) 
as the dimension of the image space f(V). If / has matrix A, then f(V) is 
the row-space of A ; that is, the subspace of V spanned by the vectors con- 
sisting of the rows of A. The rank of A is defined as the dimension of the 
row-space of A ; hence the rank of A equals the rank of /. It can be shown 
that the rank of A also equals the dimension of the column-space of A. 
The rank is unaltered by elementary transformations and can be deter- 
mined by inspection for an echelon matrix, where it is simply the number 
of nonzero rows. 

Let /be a one-to-one linear transformation of V onto V , so that /has a 
linear inverse/ -1 (Sect. 2). The spaces V, V must have the same dimen- 
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sion m and/, / l are represented by nonsingular square matrices A, B such 
that AB = BA = I, where 



I = I m = 



1 • 
1 







is the m by m identity matrix. If A is an arbitrary square nonsingular 
matrix, there exists a unique inverse A -1 such that A A -1 = I (which im- 
plies A -1 A = /). Hence B must be A -1 . The echelon matrix for a square 
nonsingular matrix A is /; the inverse A~ l may be obtained by applying 
to / the same sequence of elementary transformations that carry A into 
its echelon form I. The inverse has the properties 



(cA)- 1 = c~ l A- 



(AB)- 1 = B~ l A' 



6. DETERMINANTS, ADJOINT 

By a permutation p of the set of integers 1, 2, • • •, m is meant a function 
plk —> k' = p(k) which is a one-to-one transformation of this set onto 
itself (Ref. 2). Each such permutation is classified as even or odd according 
as the polynomials in m variables 



p = n ( x k - xd, 

k<l 



p r = n o*7>(*) - s„(i)) 

k<l 



are the same or negatives of each other. 

Example. If m = 3, and p(l) = 3, p(2) = 1, p(3) = 2, then p is even, 
since 

(xi - x 2 )(x l - x 3 )(x 2 - x 3 ) = (x 3 - xi)(x 3 - x 2 )(x 1 - x 2 ). 

One denotes by sgn p the value 1 if p is even, the value —1 if p is odd. 

The determinant (Refs. 1, 9) det A of a square m by m matrix A = (aij) 
is defined to be the scalar 



det A = Z)sgnp-a lp(1 )-a2p(2) 



' flmp(ra) 



where the sum is over all permutations p of 1, 2, • • •, m. If A is singular, 
det A = 0. For nonsingular A, det 4^0 and det A is ( — l) fc times the 
product of the scalars c appearing in the elementary transformation of type 
(1) (Sect. 4) used in reducing A to the echelon form I, where h is the num- 
ber of transformations of type (2) . 
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Let Ajj denote the submatrix of A obtained by deleting the ith. row and 
jth column. Then for any fixed i, 

in 

det A = X) ( - 1) i+i an • det A i3 - ; 
3=1 

there is an analogous result for expansion according to a fixed column j. 
One calls det A i3 - the minor of a t -y, and the expansions of det A are called 
expansions by minors. 
Example. 



a n 


Ol2 


ai3 




























«22 


a 23 




021 


«23 




a 2\ 


a 22 


«21 


a 2 2 


«23 


= an 






- «12 






+ «13 














a32 


033 




«31 


«33 




«31 


a 32 


«31 


«32 


«33 





















= CHl(ct22 a 33 ~~ a 23«32) — a 12( a 21 a 33 ~~ a 23 a 3l) 

+ ai3(«2ia32 — «22«3l). 

The adjoint (adj) A of a square matrix A is the matrix B = (&;y), where 

bn = (-1)*"+' det A yi 

(note the reversal of indices) . One has the rule 

adj A- A = (det A)- 1 
and, if det A ^ 0, 

A" 1 = (det A)" 1 -adj A, 

adj A = (det A) -A -1 . 

Cramer's Rule. If det A ^ 0, the system 



7 j a^jXj — Cj [t — i 

3 = 1 



has a unique solution 



X{ 



det A 00 
det A 



m), 



where A(i) is the matrix obtained from A by replacing the i-th column by 
c u •••, c m (Ref. 2, 9). 

7. EQUIVALENCE 

Let / be a linear transformation of V into V. It has been seen (Sect. 4) 
that the matrix A for / can be put in echelon form by a suitable change of 
basis in V. If V is not the same space as V, one can further simplify A 
by independently changing the basis for V. This effects elementary trans- 
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formations on the columns of A ; by successive subtractions of multiples of 
earlier columns from later ones, followed possibly by a renumbering of the 
basis, A can be reduced to the form 



= ( Ir °\ 

Vo 0/ 



where I r is the r by r identity matrix (Sect. 5) and the 0's stand for rows 
and columns consisting wholly of zeros; J r is a rectangular nhy m matrix, 
just as was the given matrix A. For the matrix in echelon form in the 
example of Sect. 4 the matrix J r would be 

10 

10 

10 

.0 0J 

The effect of a change of basis in V is to replace A by PA, where P is a 
nonsingular n by n matrix; the effect of a change of basis in V is to re- 
place A by AQ, where Q is a nonsingular m by m matrix. The matrix B is 
said to be equivalent to matrix A if B = PAQ for some nonsingular P and 
Q. This is a proper equivalence relation (Chap. 1, Sect. 5). The reasoning 
given above then gives the conclusion: Every A is equivalent to a unique 
matrix of the form J r . In other words, the matrices J r (for various r, m, 
and n) are a set of canonical forms under equivalence (Ref. 9). 

8. SIMILARITY 

One now considers the possible matrices A representing a linear transfor- 
mation / of the vector space V into itself. The field F of scalars will be 
assumed to be the complex number system. Since V = V, one can no 
longer change bases in V and V independently. Indeed, let a'i = Sy Pijctj 
be equations defining a new basis a\, • • ■ , a! n in V. Then P = (pij) is a 
nonsingular matrix with inverse P _1 = (g z -y), and ak = 2^ qich<x'h- Let / 
have the matrix A = (a^) relative to the basis «», so that /(«»•) = 2^ a^a'^. 
Then 

/(«'*) = £22 Vii a jkqkhOi'h, 
h j k 

and / has the matrix PAP~ l relative to the basis a'i, • • • , a f n . The square 
matrix B is said to be similar to square matrix A if B = PAP~ X for some 
nonsingular matrix P. Hence change of basis in V replaces the matrix of 
/by a similar matrix. Similarity is an equivalence relation (Chap. 1, Sect. 
5) in the class of square matrices (Ref. 9). 

If A can be reduced to a similar matrix of sufficiently simple form, most 
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of the important properties of A can be read off. The ideal situation is 
that in which A is similar to a diagonal matrix; that is, a matrix (an) in 
which a,ij = for i ^ .;'. Unfortunately, not every A is similar to a diagonal 
matrix, and the various canonical forms are approximations to the diagonal 
form. 

If A is similar to 

"Xi • • 



B = diag (Xi, ■■•,\n) = 

then in terms of the new basis «i, 
/(«i) = &\B = Xiai, 











• • X,J 
, a n associated with B one has 
, f( a n) = oc n B = \ n a n . 



In general, if a vector a t± is such that aA = \a for some scalar X, then 
X is called an eigenvalue (characteristic value, latent root) of A, and a is 
called an eigenvector belonging to X. The characteristic polynomial for A is 
the polynomial 4>(x) = det (xi — A); this is a polynomial 

4>(x) = c + c\x -\ h c n x n 

of degree n, and its n roots (real or complex) are the eigenvalues of A. In 
particular, 

(-1)% = det A = X r X 2 X„, 

— c n _i =On+--'+ a nn = Xi + • • • + X n = trace of A, and c n = 1. 

The Hamilton-Cayley Theorem (Ref . 9) states that A satisfies its char- 
acteristic equation: 

4>(A) = c I + c x A + • • • + c n A n = 0. 

If the roots of <f>(x) are distinct, then A is similar to B = diag (Xi, • • • , X n ). 
In fact, let 4>(x) be a factor of the fcth power of a polynomial \f/(x), whose 
roots are the distinct numbers Xi, • • •, X p ; if $(A) = 0, then A is similar 
to a diagonal matrix; if \^(A) ?± 0, then A is not similar to a diagonal 
matrix. 

In the general case of repeated roots, the matrix A is similar to a matrix 
B in Jordan normal form; that is, a matrix (in partitioned form, see Ref. 9, 
Sect. 2.8) 

B l 



B = diag(Bi, ••-,£,) = 



B, 



L0 






B s 
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where the Bi are square matrices of form 

\i I • 



Bi = 



X; 












Xi 1 
\i 



and Xi, • • •, X s are not necessarily distinct. In the matrix B each charac- 
teristic root X appears on the diagonal a number of times equal to its mul- 
tiplicity. 

An alternative rational canonical form for matrix A has the form B = 
diag (Bi, • • - , B p ), where B { has form 



" 


1 
























• 


1 


-— c»-i 


— Ci2 


• • 


Cikt 



If A has rational (real) entries, the B{ can be chosen so that the c# are 
rational (real). 

If A is a real matrix, the eigenvalues X need not be real but, since (f>(x) 
has real coefficients, they will occur in conjugate complex pairs. In this 
connection it is useful to note that the matrices 

/re ld \ /rcos0 — r sin 0\ 

\0 re~ t6 / \r sin r cos 0/ 

are similar. 

When A is of small degree or is otherwise especially simple, its eigen- 
values and eigenvectors can be found by explicit calculation from the defi- 
nitions given above; often they can be found from the physical interpreta- 
tion of the problem. Determination of eigenvalues is a problem in solving 
an algebraic equation (Chap. 2), but other methods are available (Ref. 4). 
If X is an eigenvalue having absolute value greater than that of all other 
eigenvalues and a is any reasonable approximation to an eigenvector be- 
longing to X (a must not lie in the subspace spanned by the eigenvectors 
of the remaining eigenvalues), then the sequence a = a.\, a 2 , • • • , a n , 
where a l+ i = a{A/c{ and C{ is the first nonzero coefficient of a; will con- 
verge to an eigenvector for X. 



MATRIX THEORY 3-13 

9. ORTHOGONAL AND SYMMETRIC MATRICES 

Let V be a real vector space, with basis a\, • • •, a n , so that each vector 
has coordinates (a\, ••-, a n ). The inner product (Ref. 9) (a, /3) of the 
vectors a = (a\, • • •, a n ), /3 = (b\, • • • , b n ) is defined as the scalar 

(a, (5) = oibi H 1- a n b n . 

The norm of a is the scalar \a\= (a, a) 1 ^. The angrfe 6 between a, /3 is de- 
fined by the equation 

(a, |8) = |a| |/3|cos0. 

These definitions are relative to the given basis but are unaffected if a 
new orthonormal basis is introduced; that is, a basis a\, • • •, a' n such that 
(a'i, a'j) = 8{j = 1 or according as i = j or i ^ j. If 

i 

then the matrix A = (a^-) has as its inverse the transposed matrix A T = 
(bij), where Oj-y = ay t -; that is, AA T = I. A matrix with this property is 
called orthogonal. Since det A = det A T , and det yl -det A T = det I = 1, 
one concludes that det ^4 = ±1. When det A = 1, A is called proper 
orthogonal and is a product of rotations; if det A = — 1, A is a product of 
rotations and one reflection, so that orientation is reversed. The eigenvalues 
of an orthogonal matrix all have absolute value equal to 1. 

A real matrix A = (aij) is termed symmetric if A = A T ; if, further, the 
quadratic form 2 t j aijXiXj is >0 except when Xi = • • • = x n = 0, then A 
and the quadratic form are said to be positive definite. The x's can be 
interpreted as coordinates of a vector a with respect to a given basis; then 
"Zij a^XiXj = (aA, a). If a new basis is chosen (not necessarily orthonor- 
mal), the form is replaced by a new quadratic form. When A is positive 
definite, the new basis can be chosen so that (aA, a) has the form SjXj 2 ; 
this is equivalent to the statement that A can be written as PP T , where P 
is nonsingular. If A is symmetric, but not necessarily positive definite, 
the new basis can be chosen so that (aA, a) has the form 

y\ 2 -\ h y r 2 - y r +\ 2 y s 2 , 

where the numbers r, s are uniquely determined by A. This is equivalent 
to the statement that there exists a nonsingular matrix P such that PAP T 
= B, where B = (b^), b {j = for i ^j, b^ = or ±1 for i — j. (One 
terms B congruent to A.) 

The eigenvalues of a symmetric matrix A are all real, and A is similar 
to a real diagonal matrix C; indeed C = PAP -1 , where P may be chosen 
to be orthogonal (Ref. 9). 
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An analogous theory holds for complex vector spaces. The inner product 
is defined as 

(a, ]8) = aibx ^ h a n b n (&»• = conj. of b { ), 

so that (a, a) = 2f|a;| 2 > 0; the norm of a is defined to be (a, a) y2 . 
Orthogonal matrices are replaced by unitary matrices, defined by the con- 
dition AA T = I, where the bar denotes replacement of each entry by its 
conjugate. Symmetric matrices are replaced by Hermitean matrices, de- 
fined by the condition A = A T . 

10. SYSTEMS OF LINEAR INEQUALITIES 

Let V be a real vector space with fixed basis {<xi, • • •, a n } as in Sect. 9, 
so that each vector a has coordinates (cti, • • • , a n ). If the vector /3 has 
coordinates (bi, • • •, b n ) then one writes 

/3 > a or a < /3 if ai < b { (i = 1, • • •, n), 

/3 ^ a or a 52 /3 if a{ ^ b{ (i = 1, • • • , n), 

/3 > a or a < /3 if a ^ but a ^ 0. 

The relation ^ is a partial order; the relations < and < are antisymmetric 
and transitive but not reflexive (Chap. 1, Sects. 4 and 7). A vector a is 
said to be 

non-negative if a ^ 0, 
positive if a > 0, 
strictly positive if a. > 0. 

The set Q of all non-negative vectors is called the positive orthant in V. 
A positive vector a such that a\ -\- •■•-{- a n = 1 is called a probability 
vector. 

For fixed a and real number k, the set of all vectors £ = (#i, • • •, x w ) 
such that 

(a, £) + k = aiXi -\ 1- a n x n + A; ^ 

is a closed set (Chap. 1, Sect. 8) called a half-space 3C. For example, the 
solutions of 2x\ + 3x 2 —6^0 form the half-space (half-plane) in two- 
dimensional space, as shaded in Fig. 1. Similarly, the solutions of {a, £) -f- 
k > constitute an open half-space 3C . The solutions of {a, £) + k = 
constitute a hyperplane IT which is the boundary of both 3C and 3C . By a 
system of linear inequalities is meant a set of relations 

(a K , £) + k K R K 0, 
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where' the index k ranges over a given set (possibly infinite), and for each n, 
R K is one of the relations > , ^ , = . For example; 



2xi + 3rr 2 - 6 ^ 0, 



xi -f- 5x 2 > 0, 



xi + x 3 = 



is a system of linear inequalities. By a solution of the system of inequali- 
ties is meant a vector £ = (xi, • • •, rc n ) which satisfies all the inequalities. 




Fig. 1. Half-space in two dimensions. 

With each inequality is associated a half-space (or hyperplane) 3C K . The 
set of all solutions of the system is the intersection of all 3C K . 

Convexity. The vector a is said to be a convex combination of vectors 



' , «m if 



a = piai H h p OT a m , pi H h p,« = 1, Pi ^ (i = 1, • • •, m). 

A nonempty set /C in V is said to be convex if it contains all convex combina- 
tions of its vectors. If K is interpreted as a point set in n-dimensional 
space, K is convex if and only if, for each pair of points a\, a 2 in K, the 
line segment joining ai to a 2 lies in K. (See Fig. 2.) A half-space 3C is 
said to be a support for a convex set K in F if iT is a subset of JC. If, 
moreover, IT contains n — 1 independent vectors of K, 3C is called an ex- 
treme support for K. 

Let T be a subset of V. The set of all convex combinations of vectors 
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in T is a convex set, called the convex closure of T. A convex set K is said 
to be finitely generated if it is the convex closure of a finite subset of K. 




Fig. 2. Convex set. 



A set T is said to be bounded if, for some constant M , 
I «i H h | a n | ^ AT f or all a in T. 

Double Description Theorem. // K is a finitely generated convex set 
in V, then K is the intersection of a finite number of supports; moreover, if K 
spans V, then K is the intersection of its extreme supports. Conversely, if the 
intersection of a finite collection of half-spaces is nonempty and bounded, then 
it is finitely generated. 

This theorem, when formulated in algebraic terms, is known as Farkas' 
Lemma: 

Farkas' Lemma (strong nonhomogeneous form). Let V be n-dimen- 
sional real vector space, let V be w-dimensional real vector space; fixed 
bases are assumed chosen in each. Let A be an n by m matrix, let 8 be a 
vector in V , let k be a scalar, and suppose that there is at least one vector 
<f> in V such that <j>A ^ 5. Then a vector a in V will satisfy the condition 
(a, <f>) ^ k for all 4> for which 4>A ^ 8 if and only if there exists a vector 
7 ^ in V such that a = yA T and (y, 8) ^ k. 

Farkas' Lemma (weaker homogeneous form, k = 0, 8 = 0). Let A be 
an n by m matrix. Then a vector a in 7 will satisfy the condition 
(«, 4>) ^ for all <f> for which <f>A ^ if and only if there exists a vector 
7 ^ in V such that a = yA T . 

Farkas' Lemma can be used as a foundation for the Minimax Theorem in 
game theory and for the Duality Theorem in linear programming. These 
theorems can also be deduced from the following one: 
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Theorem. Let A = (a^-) and B be n by m matrices with aij > Ofor all i, j. 
Then there exist probability vectors £ in V a?id 77 in V and a unique scalar k 
such that 

(kA - B)r) ^ and £(L4. - B) ^ 0; 

in the first inequality n is regarded as an m by 1 matrix. 
For all of Sect. 10, see Refs. 5, 6, 9. 
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1. DEFINITIONS 

By a difference equation is meant an equation relating the values of an 
unspecified function / at x, x + h, x -f- 2h, • • • , x + nh, where h is fixed. 
For example, 

(1) fix + 3) - f(x + 2) - xf(x + 1) - 2f(x) = x 2 

is a difference equation, in which h = 1, n = 3. The variable x will gen- 
erally be assumed to vary over the discrete set of real values x + ph 
(p = 0, ±1, ±2, • • •), where x is a constant. By proper choice of origin 
Xq and scale one can make x = 0, h = 1, so that x varies over the integers 
0, ±1, ±2, • • •. In the subsequent discussion, this simplification will be 
assumed made, so that x varies over the integers and the difference equa- 
tion thus relates the values of / at x, x + 1, • • •, x + n. For the case 
when x varies continuously, see Remark at the end of Sect. 4. The values 
of / are assumed to be real, although much of the theory extends to the 
case in which / has complex values. 
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A general difference equation is constructed from a function \f/(x, y , y 1} 
■ • • 5 Vn) of the integer variable x and the n + 1 real variables y , • • • , y n . 
The difference equation is the equation 

(2) f(x, f(x), f(x + 1), • • • , f(x + n)) = 0. 

By a solution of the difference equation is meant a function / which satisfies 
it identically. When the equation takes the form 

(3) fix + n) = 4>ix, fix), -••,f(x + n- 1)) 

and 4>ix, y , • • •, y n -i) is defined for all values of x,y , • • •, y n —\, eq. (3) is 
simply a recursion formula. If /(0), /(l), • • •, /(n — 1) are given arbitrary 
values, then eq. (3) determines successively fin), fin +1), • • • ; thus there 
is a unique solution for x ^ with the given initial values /(0), /(l), • • •, 
fin - 1). 

The ./Ersf difference of /(#) is A/ = fix + 1) — fix); the second difference 
is A 2 / = A(Ajf) = f{x + 2) - 2/(s + 1) + fix); the &th difference is A k f. 
A difference eq. (2) can be written in terms of / and its differences. For 
example, eq. (1) is equivalent to the equation 

(1') A 3 / + 2A 2 / + (1 - x)Af - (3 + x)f = x 2 . 

Conversely, an equation relating/, A/, • • •, A n f can be written in form (2). 
Thus, eq. (2) is the general form for difference equations, and this form 
will be used throughout this section, in preference to an equation relating 
the differences of /. 

The order of the difference eq. (2) is defined as the distance between the 
most widely separated ^-values at which the values of / are related. If \f/ 
definitely depends on fix) and fix + n), then the order is n. However, 
the order may be less than n. For example, the equation 

(4) fix + 4) - 2fix + 3) - f(x + 1) = 

has order 3, since the most widely separated values are x + 1 and 3 + 4. 
The substitution gix) = fix + 1) reduces this to an equation relating gix), 
gix + 2), gix + 3). 
Operator Notation. If y is a function of x, one writes 

(5) E k y = yix + k) (fc = 0, 1, 2, • • •). 

Thus E°y = yix), E x y = Ey = yix + 1). The difference eq. (2) can thus 
be written 

(20 +ix,y,Ey,E 2 y, • • • , E n y) =0. 
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2. LINEAR DIFFERENCE EQUATIONS 

By a linear difference equation is meant an equation of form 

(6) a n f(x + n) + a„_i/(x + n - 1) -\ h a x j{x + 1) + a f(x) = v(x), 

where a , • • • , a n , v(x) are given functions of the integer variable x. In 
terms of the operator E of Sect. 1, the equation can be written: 

(6') a n E n y -\ 1- a x Ey + a y = v(x), 

where y = f(x). It can be written more concisely as follows: 

(7) HE)y = »(*), 
where ${E) is a linear difference operator: 

(8) <K#) = a n E n +---+ ai E + a . 

If y(x) = 0, eq. (6) is termed homogeneous; otherwise it is nonhomogeneous. 
In case ao ^ 0, a n ^ 0, the equation is of order n. In case a = a± = • • • = 
a m _i = 0, but a m a n ^ 0, then it is of order n — m; the substitution g(x) = 
f(x — m) then reduces eq. (6) to a linear equation for g of form (6), with 
nonvanishing first and last coefficients. 

Linear Independence. Let yi(x), • • •, y p (x) be functions of x defined 
for a < x < b. The functions are said to be linearly independent if a relation 

hyifr) H h b p y p (x) = 

with constant ^i • • • , b p , can hold only if t>i = b 2 = • • • = b p = 0. Other- 
wise, the functions are said to be linearly dependent. 

General Solution. Let the difference eq. (6) be given, with v(x) =0 
and ao(x)a n (x) ^ for a < x < b\ all coefficients are assumed defined for 
a < x < b. Then the equation has order n, there are n linearly inde- 
pendent solutions ?/i (x), • • • , y n (x) for a < x < b and 

(9) y = c x y x (x) -\ \- c n y n {x), a < x < b, 

where c\, ■ • -, c n are arbitrary constants, is the general solution; that is, 
all solutions are given by eq. (9). If v(x) ^ 0, but the other hypotheses 
hold, then the general solution has form 

(10) y = cit/iO) H h c n y n (x) + V(x), 

where V(x) is a solution of the nonhomogeneous equation and Ciyi(x) + 

h c n y n (x) is the general solution of the related homogeneous equation, 

that is, the homogeneous equation obtained by replacing v(x) by 0. 

Example. The functions yi = 1, y 2 = x are linearly independent solu- 
tions of the equation (E 2 — IE + \)y = 0, so that y = Ci + c 2 x is the 
general solution; y = 2 x ~ l is a solution of the equation (E 2 — 2E + l)y = 
2 X ~ X , so that y = ci -\- c 2 x + 2 X— x is the general solution. 
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3. HOMOGENEOUS LINEAR EQUATIONS WITH CONSTANT 
COEFFICIENTS 

The equations considered have form 

(11) HE)y = 0, 
where 

(12) HE) = a n E n + • • • + ai#i + a , 

the coefficients a n , • • • , a are constants, and a a n ^ 0. Associated with 
eq. (12) is the characteristic polynomial 

^(X) = a n \ n H h a{k + a 

in the complex variable X. The equation 

(13) *(X) = 

is an algebraic equation of degree n, called the characteristic equation or 
auxiliary equation associated with eq. (11). The characteristic equation 
has n roots Xi, • ■ •, X n called characteristic roots. (See Chap. 2.) These 
may be real or complex; since the coefficients are assumed real, the complex 
roots come in conjugate pairs. 

From the set of characteristic roots one obtains a set of n solutions of the 
difference eq. (11) by the following rules: 

I. To each simple real root X one assigns the function X*; 

II. To each real root X of multiplicity k one assigns the k functions \ x , 
x\ x , ••-,x k - l \ x ; 

III. To each pair of simple complex roots a -f- j3i = p(cos ±i sin 4>) 
one assigns the functions p x cos <f>x, p x sin <f>x; 

IV. To each pair of complex roots a ± $i = p(cos <p ± i sin 0) of mul- 
tiplicity k one assigns the 2k functions 

p x COS (f)X, Xp x COS (f)X, • • • , x k ~ 1 p x cos 4>x, 

p x sin 4>x, xp x sin cf>x, • • • , x k ~ 1 p x sin 4>x. 

In all one obtains n functions y\{x), • • • , y n (x) which are linearly independ- 
ent solutions of eq. (11) for all x, so that 

y = ciyi(x) H h c n y n (x) 

is the general solution. 

Example. (E* - 8E 3 + 25E 2 - 3QE + 20)y = 0. 
The characteristic equation is 

X 4 - 8X 3 + 25X 2 - 36X + 20 = 0. 
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The roots are 2,2,2 dc i. Hence the general solution is 

y = 2 x (ci + c 2 x) + 5 xl2 (c 3 cos <f>x + c 4 sin <j>x), 

where <£ = arctan Yi- 

Remarks. The variable x has heretofore assumed only integral values. 
If x is allowed to take on all real values, then the difference equation be- 
comes a functional equation. The methods of this section are still applica- 
ble and provide the general solution of eq. (11) subject only to the follow- 
ing two modifications: (a) the arbitrary constants C\, c 2 , • • • may be re- 
placed by arbitrary periodic functions of x, of period 1 ; (6) if X is a negative 
characteristic root of multiplicity k, the corresponding solutions become 

( — \) X COS TTX, X{— X) X COS TTX, • • •, X k ~ l ( — X)* COS 1TX. 

A. NONHOMOGENEOUS LINEAR EQUATIONS WITH CONSTANT 
COEFFICIENTS 

The equation considered is 

(14) *(E)y = v(x), 

where \f/(E) satisfies the same conditions as in Sect. 3. By the rule stated 
at the end of Sect. 2, the general solution of eq. (14) has the form 



(15) 



V = ciyiix) -\ \- c n y n (x) + V(x), 



where V(x) is a particular solution and the other terms are the general solu- 
tion of the related homogeneous equation \fs(E)y = 0. 

The procedures for finding the particular solution V(x) can be described 
concisely by means of an operational calculus which parallels that used for 
differential equations (Chap. 8). The operators ip(E) with constant coeffi- 
cients can be added, subtracted, multiplied, and multiplied by constants 
just as polynomials. The operators can be converted into operators x(A) 
by the relation 

(16) A = E - 1. 

For example, 

E 2 - 1 = (A + l) 2 - 1 = A 2 + 2A. 

The powers A, A 2 , • • • , A k are the first, second, • • • , fcth difference, as de- 
fined in Sect. 1. 

If y = V(x) is a solution of eq. (14), one writes 

(17) V(x) = —=rv(x) = [tm-'vix). 

HE) 
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Table 1. Rules for Particular Solutions 

No. f(E) v(x) W(E)]-h 

1. yp{E) civiix) + c 2 v 2 (x) ; ci, c 2 const. ci[yp{E)}~ l v x + c 2 [rA(^)J -1 y 2 

2.UE)UE) »(x) sg)(di)") 

x-1 

3. A = E - 1 K*), x = 0, ±1, • • • A-!y = £ t>(A) 

fc=0 

4. a 2 = # 2 - 2# + l v(x), x = o, ±i, • • • A- 2 y = E z. <*) 



• ix(x- i) ••• i 

5. A' = (*-!)' H=-i (a: ~, n+1)[ A-H = ( 1) 

\n/ n! \n/ \n + r/ 



6. ME) a x u{x) a x ,. _. w(x) 

^(a#) 

7. *(£) a*,#i)^0 ^y 

A; x— & 

8. (# - a) k <KE) a x , 0(a) ^ * a 



cf)(a)k 



9. i/<#) a x w(x), i/<a) ^ 0, w a poly- a x [p(a) + ^ p'(a)A + • 

nomial of degree s 

+ -^ p s (a)A s ]u(x), 

p(X) = 1/*(X) 

10. (i? — a) k (j)(E) a x u{x), u{x) a polynoial of a x - k [q(a)A- k 

0(a) t^ degree s 

+ -g'(a)A 1 -*+--- 

a s 
+ — q s (a)A s - k ]u(x), 

q(\) = 1/0(X) 

a *-l^-l( a -ar y ) 

(p*-Vi8)[sin(0a;-0). 
A -1 (p _x cos <f>x v) 

— COS (0X ' — 4>) • 
A~\p~ x smcf>xv)] 



1. E - a 




?;(a;) 


2. (# - af 




v(x) 


3. (#-a) 2 + |8 2 




v(x) 


a + t/3 = p(cos 


4> 




+ i sin 0) 
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Thus the inverse operator [${E)]~ l , when applied to v(x), yields one solu- 
tion of the eq. (14). The rules for finding particular solutions can now be 
summarized in a table, which evaluates [${E)]~ l v for various choices of \f/ 
and v. This is carried out in Table 1. The last column gives one choice 
V(x) of [\p(E)]~ 1 v; the general solution is given by eq. (15). 

The binomial coefficient (*) of Rule 5, Table 1, is defined for n = 1,2, 
When n = 0, it is defined to equal 1, and Rule 5 remains valid. Corre- 
sponding to this inverse rule is the direct rule: 

(18) A'(*) = ( B I r ), < r ^ n 

= 0, r > n. 
A general power* of x can be expanded in terms of these coefficients: 

(19) x n = TVUOG) + • • • + r„ B (ni)(lD, 

where the Tj are Stirling numbers of the second kind. They are tabulated 
on page 170 of Ref. 2. If the polynomial u{x) is expanded in terms of the 
coefficients by eq. (19) and Rule 5 or eq. (18) is applied, then Rules 9, 10 
are easier to use. 

A general expression l/\p(E) can be regarded as a rational function of E 
and expanded in partial fractions, just as if E were a numerical variable. 
Rules 11, 12, 13 then permit evaluation of the terms. For example, 

1 1 / 1 1 \ 

v(x) = v(x) = [ ) v(x) 



E 2 - 3E + 2 (E - l)(E - 2) \E - 2 E - 1 

= 2 x - 1 A~ l (2~ x v) - A~h. 

Rule 12 is needed for multiple roots. Rule 13 is needed for complex roots; 
it can be generalized to take care of repeated complex roots (Ref. 2). 

5. LINEAR EQUATIONS WITH VARIABLE COEFFICIENTS 
The general solution of the first order linear equation 

(20) [E - V (x)]y = v{x), x^a, 

where p(x) ^0 for x ^ a, is 

7 a q(s+ 1). 
(22) q(x) = p(a)p(a + 1) • • • p(x - 1), x > a 

= 1, x = a. 



(21) y = q(x) 



+ Z 
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Laplace's Method. For equations 

(23) [a n (x)E n + • • • + ai (x)E + a (x)]y = 
with polynomial coefficients, one seeks a solution 

(24) y(x) =■ f t x ~ l v(t) dt, 

J a 

where a, b, and v(t) are to be determined. 

Let (x) n denote n\Q), so that (x) n = x{x — 1) • • • (x — n + 1) for n = 
1, 2, • • •, and let (x) = 1. It follows from eq. (19) that an arbitrary poly- 
nomial can be expressed as a linear combination of the polynomials (x) n . 
Hence the coefficients dk(x) can be considered as linear combinations of the 
{x) n . Now by integration by parts one obtains from eq. (24) the relation 

(25) {x + m- l) m E r y 

m "1 & 

£ (-l) s+1 (a; + m - l) m _F- 1+ 'D— M^MOl 

s=l Jo 

+ (-l) m f ^ +w - 1 D w {f n «;(0} dt, 

J a 

where D s = d s /dt s . Hence the difference eq. (23) takes the form 

h r b 

(26) [F{x, v, t)] + t x ~ l G{v, t) dt = 0. 

a Ja 

The function v{t) is chosen so that G(v, t) =0. In fact, the equation 
G{v, t) = is usually a homogeneous linear differential equation for v(t). 
The constants a and b are then chosen so that F(x, v, t) vanishes when 
t = a and t = b, so that eq. (26) is satisfied. Once a, b and v(t) have been 
determined in this way, eq. (24) then yields y(x). 
For further details see Ref. 2, Sect. 174. 
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1. BASIC CONCEPTS 

An ordinary differential equation is an equation of form 

(i) Hx,y,y', •••,2/ (n) ) =0, 

expressing a relationship between an unspecified function y of x and its 
derivatives y' = dy/dx, • • •, y (n) = d n y/dx n . An example is the following: 

y' — xy — 0. 

The order of the equation is n, which is the order of the highest derivative 
appearing. A solution is a function y of x, a < x < b, which satisfies the 

5-01 
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equation identically. For many equations one can obtain a function 

(2) y = f{x, c x , •••, c„), 

expressing y in terms of x and n independent arbitrary constants c\, • • • , c n 
such that, for each choice of the constants, eq. (2) is a solution of eq. (1), 
and every solution of eq. (1) is included in eq. (2). When these conditions 
are satisfied, eq. (2) is called the general solution of eq. (1). A particular 
solution is the general solution with all of the n arbitrary constants given 
particular values. 

If eq. (1) is an algebraic equation in y (n) of degree k, then the differential 
eq. (1) is said to have degree k. For example, the equation 

(3) y'" 2 + y"y'" + 2/ 4 = e* 

has order 3 and degree 2. When the degree is 1, the equation has the form 

(4) p(x, y, -•-, y {n - x) )y {n) + q{x, y, • • •, y {n ~ l) ) = 
or, where p^O, the equivalent form 

(5) y<»> = F(x,y, ■■-,y^- 1) ), F = -q/p. 

The Existence Theorem asserts that, if in eq. (5) F is continuous in an 
open region R of the space of the variables x, y, • • -, y^ n ~ l) , and (x , yo, • • •, 
y Q (.n— i)) z - g a p i n i j r^ {\ ien there exists a solution y(x) of eq. (5), \x — x \ < h, 
such that 

(6) y = y , y' = y'o, ■•-, 2/ (n_1) = y (B_1) for x = x . 

Thus there exists a solution satisfying initial conditions (6). If F has con- 
tinuous partial derivatives with respect to y, y', • • • , y {n ~ l) in R, then the 
solution is unique. 

2. EQUATIONS OF FIRST ORDER AND FIRST DEGREE 

An equation of first order and first degree can be written in either of the 
equivalent forms 

(7) y' = F(x, y), 

(8) M(x, y) dx + N(x, y) dy = 0. 

For equations of special form, explicit rules can be given for finding the 
general solution. Some of the most important types are listed here. 

Equations with Variables Separable. If in eq. (8) M depends only 
on x, N only on y, then eq. (8) is said to have the variables separable. The 
equation may then be written with the x's separated from the y's, and the 
general solution may be obtained by integration. 
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Example, y' = 3x'"y. An equivalent separated form is 3x 2 dx — y~ x dy 
= 0. Hence 



j 3x 2 dx — I y 1 dy = 



Integrating and solving for y one finds y = c^ as the general solution, 
where C\ — e~ c . 

Homogeneous Equations. A function F(x, y) is said to be homogene- 
ous of degree n if F(Xx, \y) = X n F(x, y). The differential eq. (7) is said 
to be homogeneous if F(x, y) is homogeneous of degree 0. To solve such 
a differential equation write y = vx and express the differential equation in 
terms of v and x. The resulting differential equation has variables separable 
and can be solved as above. In general, y' = F(x, y) becomes 

xv' + v = F(x, vx) = x°F(l, v) = G(v), 

dx dv 

— H = 0. 

x v — G(v) 

Exact Equations. The differential eq. (8) is exact if for some function 
u(x, y) 

du du 

(9) — = M(x, y), — = N(x, y), 

ox ay 

so that du = M dx + N dy. The equation is exact if and only if dM/dy = 
dN/dx. The general solution is given (implicitly) by u(x, y) = c. 

Example. (3a; 2 - 2xy) dx + (2y - x 2 ) dy = 0. Here dM/dy = -2x 
= dN/dx, so that the equation is exact. Then 

du du . 

— = 3x 2 - 2xy, — = 2y - x 2 . 

dx dy 

From the first equation, y = x 3 — x 2 y + g{y), where g{y) is an arbitrary 
function of y. Substitution in the second equation yields the relation 
—x 2 -\- g'{y) = 2y — x 2 , so that g(y) = y 2 + c. Hence the general solu- 
tion is x 3 — x 2 y -\- y 2 — c. 

Integrating Factors. If the eq. (8) is not exact, it may be possible to 
make it exact by multiplying by a function <j>(x, y), called an integrating 
factor. 

Example. The equation (3xy + 2y 2 ) dx + (x 2 + 2xy) dy = is not 
exact, but after multiplication by x becomes the exact equation 

(3x 2 y + 2xy 2 ) dx + (x 3 + 2x 2 y) dy = 0. 
The general solution is x 3 y + x 2 y 2 — c. The integrating factor is x. 
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Linear Equations. A differential equation is linear if it is of the first 
degree in the dependent variable and its derivatives. If such an equation 
is also of the first order, it may be written in the form 

(10) y' + p(x)y = q(x). 

Here u = & v x is an integrating factor and the general solution is 

(11) y = u~ 1 I I Q(x)udx + c), u = e JP *. 
Example, y' + x~ x y = 4a; 2 . Here u = x and eq. (11) gives 

y = x~ l f I 4x 3 dx + c J = x 3 + ex -1 . 

3. LINEAR DIFFERENTIAL EQUATIONS 

The linear differential equation of order n can be written in the form 

(12) a D n y + a^^y + • • • + a n _ x l>y + a n y = Q(x), 

where the coefficients a , • • • , a n may depend on x, and D k y = d k y/dx k . 
When the ay are constant, eq. (12) is said to have constant coefficients. 
When Q(x) = 0, the equation is said to be homogeneous. The homogeneous 
equation obtained from eq. (12) by replacing Q(x) by is called the related 
homogeneous equation. It will generally be assumed that a ^ throughout 
the interval of x considered. 

The general solution of eq. (12) is given by 

(13) y = ctyiix) -\ h c n y n (x) + y*(x), 

where y*(x) is one particular solution and y\(x), • • • , y n (%) are particular 
solutions of the related homogeneous equation which are linearly inde- 
pendent; that is, a relation 

biyi(x) -f h 2 y 2 (x) -\ \- b n y n (x) = 0, 

with constant &i, • • • , b n can hold only if &i = 0, • • -, b n — 0. When 
Q(x) = 0, one can choose y*(x) to be 0. 

Homogeneous Linear Equations with Constant Coefficients. The 

equation has the form 

(14) a D n y -\ 1- o n _iZ)y + a n y = 0, a Q * 0, 

where ao, • • • , a n are constants. Particular solutions are obtained by set- 
ting y = e rx . Substitution in eq. (14) leads to the equation for r: 

(15) a r n -{ r- On-ir + a n = 0. 
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This is called the auxiliary equation or characteristic equation. In general 
it has n roots, real or complex, some of which may be coincident (Chap. 2). 
From these roots one obtains n linearly independent solutions of the dif- 
ferential eq. (14) by the following rules: 

I. To each real root r of multiplicity k one assigns the functions e kx , 

II. To each pair of conjugate complex roots a ± (Hi of multiplicity k one 
assigns the 2k functions 

e ax cos fix, e ax sin fix, xe ax cos fix, xe ax sin fix, 

••', x k ~ 1 e ax cos fix, x k ~ 1 e ax sin fix. 

The n function y\(x), • • • , y n {x) thus obtained are linearly independent and 

y = c x y x (x) + • • • + c n y n (x) 

is the general solution of eq. (14). 

Example 1. D 2 y — 3Dy + 2y = 0. The auxiliary equation is r 2 — Sr 
+ 2 = 0, the roots are 1, 2; the general solution is y = c x e x + c 2 e 2x . 

Example 2. D 6 y - 9D 4 y + 2W 2 y - IQy = 0. The auxiliary equa- 
tion is r 6 — 9r 4 + 24r 2 — 16 = 0, the roots are ±1, ±2, ±2; the general 
solution is y = c x e x + c 2 e~ x + e 2x (c 3 + c 4 x) + e~ 2x (c 5 + c 6 x). 

Example 3. D 4 y + 4D 3 y + 12D 2 y + lQDy + 16 = 0. The auxiliary 
equation is r 4 + 4r 3 + 12r 2 + 16r +16 = 0, the roots are — 1 ± i\/3, 
— 1 ± i-\/3. The general solution is y = e~ x [(c x + c 2 x) cos \/S x + 
(c 3 + c 4 x) sin -y/Sx]. 

Nonhomogeneous Linear Equations with Constant Coefficients. 
The equation considered is 

(16) a D n + • • • + a n ^Dy + a n y = Q(x), a ^ 0, 

where ao, • • •, a n are constants and Q(x) is, for example, continuous for 
a < x < b. The general solution of the related homogeneous equation is 
found as in the preceding paragraphs; it is called the complementary junc- 
tion. Here are presented methods for finding a particular solution y*(x) 
of eq. (16). As indicated in eq. (13), addition of the complementary func- 
tion and y*(x) gives the required general solution of eq. (16). 
Method of Undetermined Coefficients. If Q(x) is of form 

(17) e ax [p(x) cos fix + q{x) sin fix], 

where p(x) and q(x) are polynomials of degree at most h, then there is a 
particular solution 

(18) y* = x k e ax [<i>{x) cos fix + \j/(x) sin fix], 
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where <f>(x) and -^(x) are polynomials of degree at most h and a ± fit is a 
root of multiplicity k (possibly 0) of the auxiliary equation. If /3 = 0, Q is 
of form e ax p{x) ; also p and g may reduce to constants (h — 0). The coeffi- 
cients of the polynomials $, ^ can be considered as undetermined coeffi- 
cients; substitution of eq. (18) in eq. (16) leads to relations between these 
coefficients from which all can be determined. As an example consider the 
equation 

(Z> 2 + \)y = 3 cos x. 

Here a = 0, /3 = 1, h = 0. Since dti are roots of the auxiliary equation, 
k = 1 and 

y* = x(A cos # + B sin x). 

Substitution in the differential equation leads to the relation 

2( —A sin x + 5 cos #) = 3 cos a;. 
Hence B = %, A = 0; y* = %x sin x and the general solution is 

y = \ x sin x + Ci cos x + c 2 sin x. 

Superposition Principle. If in eq. (16) Q(x) is a linear combination 
of functions Qi(x), • • •, Qn(%) and 2/i*(x), • • •, yN*(%) are particular solu- 
tions of the respective equations obtained by replacing Q(x) by Q\(x), • • •, 
QjvOk), then the corresponding linear combination of yi*(x), • • •, yN*(x) 
is a solution of eq. (16) ; that is, if 

Q(x) =hQ 1 (x)+---+b N Q N (x), 
then 

y*(x) = hiys*{x) -1 1- b N y N *(x) 

is a particular solution of eq. (16). For example, particular solutions of 

(D 2 + l)y = 3 cos a, {D 2 + l)y = 5e 2 * 

are found by undetermined coefficients to be %x sin x, e 2x respectively. 
Hence a particular solution of 

(D 2 + l)y = 12 cos x + 10e 2 * 

is given by Qx sin x + 2e 2x . 

Variation of Parameters. Let the complementary function be 

y = c x yx{x) H h c n y n {x). 

Then a particular solution is 

(19) y* = «i(aOyi(x) H r- ^OO^/nOr), 
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where 



Vi( x ) — I Wi(x) dx, • • •, v n (x) = I w n (x) dx 



and Wi(x), • • •, w n (x) are denned by the linear equations 

yi(x)wi(x) H h ?/ n (x)w n (a;) = 0, 

2/'i(a;)ii>i(a;) -\ \- y' n (x)w n (x) = 0, 

(20) 

f/i^-^MCc) +• • •+ y n (n ~ 1) (x)w n (x) = Q(x)/ao. 
The determinant of coefficients of eqs. (20) is the Wronskian determinant 

2/i (z) • • • Vn{x) 

y'i(x) • • • y'nix) 



(21) 



W = 



2/i 



(n-D 



(X) 



1/n 



(n-1) 



(X) 



Under the assumptions made, W cannot equal for any x of the interval 
considered, so that eqs. (20) have a unique solution (Chap. 2). This 
method is applicable if a , • • • , a n are functions of x, provided ao(x) ^ 0. 
Operational Methods. The operational methods based on the Heavi- 
side calculus provide another powerful tool for obtaining solutions of non- 
homogeneous linear equations with constant coefficients (see Chap. 8). 
Closely related are the methods based on the Laplace transform (Chap. 9). 

4. EQUATIONS OF FIRST ORDER BUT NOT OF FIRST DEGREE 
The equations considered have form 

(22) f(x, y, v) = 0, 

where p = dy/dx. Equation (22) can be solved for p, except where 4> v = 0. 
The locus defined by the two equations 



(23) 



f(x, y, p) = 0, 



fpfa y, p) = 



is called the singular locus. It may contain curves y = f(x) which are 
solutions of eq. (22) ; such solutions are called singular solutions. The solu- 
tions of eq. (22) (with the possible exception of the singular solutions) can 
often be obtained by one of the following special methods. 
Factorization. If eq. (22) can be factored in the form 

(24) [p - F x {x, y)][p - F 2 (x, y)] • • • [p - F k (x, y)] = 0, 
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then its solutions are obtained by combining all solutions of the first degree 
equations 

(25) p = Fi(x, y), •■-, p = F k (x, y) (p = dy/dx). 
For example, the equation 

p 2 - (2x + y)p + 2xy = 

can be factored into the equations 

p = 2x, p = y; 

the solutions are y = x 2 + c\, y = c 2 e x . If \p(x, y, p) is of second degree in 
p, the expressions for p and the equivalent factorization (24) can be ob- 
tained by the quadratic formula. 

Solving for y or x. If eq. (22) is of first degree in y, one can solve for y 
to obtain an equation 

(26) y = F{x, p). 

Differentiation of this equation with respect to x yields a relation of form 

\dp 

(27) -f = G(x,p), 

dx 

that is, a first order equation relating p and x. If the general solution of 
eq. (27) is given by 

(28) 4>(x, p) = c, 
then the equations 

(29) y = Fix, p), cf>(x, p) = c 

together define solutions of eq. 22; p may be eliminated between the equa- 
tions or treated as a parameter. As an example, consider the Clairaut 
equation: 

(30) y = xp + F(p). 

The method described leads to the "general solution" 

(31) y = ex + F(c). 

There is, in general, a singular solution defined by the equations 

(32) x + F'(p)=0, y = xp + F(p). 

If the eq. (22) is solvable for x, one can differentiate with respect to y, re- 
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placing dx/dy by l/p; one obtains the solutions in the form 

(33) <Ky,p)=c, y = F(x,p). 

5. SPECIAL METHODS FOR EQUATIONS OF HIGHER THAN FIRST ORDER 

Equations with Dependent Variable Missing. Let the given equa- 
tion be 

(34) F(x,y',y", ■■-,y w ) = 0, 
so that y does not appear. Set p = dy/dx. Then 

dp d 2 p 

y" = -L, y"> = -JL, ... f 

dx dx 

and so eq. (34) becomes 

dp d n ~ l p 

dx dx 



(34') F[x,p,-^ ...,-_) = 0, 



an equation of order n — 1 for p in terms of x. If its solutions are known, 
then the solutions of eq. (34) are obtained from the relation y = I p dx. 
Example. Consider the equation 



x*y" - x 2 y' = 3 + x 2 . 

The substitution p = y' leads to the first order linear equation 

dio 

x 6 x 2 p = 3 + x 2 . 

dx 

Its general solution is found (Sect. 2) to be 

1 
p = + 1 + CiX. 

x z 

Hence integration yields y : 

1 l 2 

y = - + x 4- - ciar + c 2 . 
x 2 

Equations with Independent Variable Missing. Let the given 
equation be the nth order equation 

(35) F(y,y',y",-'-,y M ) =0, 
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so that x does not appear. Set p = y' . Then 
d 2 y dp dp dy dp 
dx 2 dx dy dx dy 

d s y _ d 2 p /dp x 2 



= P 2 — L + 
ete 2 dy 2 \dy 

Thus eq. (35) becomes an equation of order n — 1. If its solutions are 
known, in the form 

V = 4>(y,ci, •■•,c„_i), 
then 

dy ^y 

— = <£, = dx. 

dx <4>(y, ci, • • •) 

Thus integration yields an implicit form of the solutions of the given equa- 
tion. 

Linear Equations with One Known Solution. Let a linear equation 
be given: 

(36) a (x)y w (x) + • • • + a n (x)y = Q(x). 

Let y\{x) be a solution of the related homogeneous equation. Then the 
substitutions 

(37) y = yi(x)v, w = v' 

leads to an equation of order n — 1 for w. If w has been found, integration 
and multiplication by y\(x) yields y. 

6. SOLUTIONS IN FORM OF POWER SERIES 

Formation of Taylor Series. Let an equation of order n be given: 

(38) y M =F(x,y,y',..-,y^-») 

and let F be expressible in an absolutely convergent power series in powers 
of x, y, y', • • • for \x\ < a, \y\ <bi, ■ • • , |y (n_1) | < b n , so that F is an 
analytic function of the n + 1 variables. Then the solution f(x) of eq. (38) 
with initial conditions: y = 0, • • •, y (n— x) = at x = is expressible as a 
power series in x for |a;| < p, provided p is sufficiently small. The series 
is the Taylor series of f(x) : 

J (fc) (0) 

(39) /(*). = /(0) + z/'(0) + • • •+ ** -77-^ + • • '• 

fc! 

The values of /(0), • • • , / (n_1) (0) are given to be 0. The values of f {n) (0), 
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/ (n+1) (0), • • • are obtained from the differential equation: 

f M (0) = F(0,0, ...,0), 

f n+l) (x) = F x + F v y' + • • • + />-V B> , ■--, 

/(»+« (0) = ^(0, • • -, 0) + *>-i>F(0, • • ■, 0), • • -. 

If the initial value £o of re is not 0, one can introduce a new independent 
variable xi = x — x . If the initial values y , • • •, of y, • • •, 7/ (n_1) are 
not 0, one can introduce a new dependent variable 



/ rc n_1 \ 

u = y-(y + y' x + • ■ • + y Q {n ~ l) — ; 

\ (n — 1) !/ 



then the initial values of u, u', • • • , w (n_1) are 0. 

Special Methods for Linear Equations. For simplicity attention 
will be restricted to the second order equation 

(40) y" + p(x)y' + q{x)y = 0. 

If p(x) and q (x) are analytic at x = 0, then x = is called an ordinary 
point of eq. (40). The Taylor series solution eq. (39) can then be obtained 
as above. One can also substitute a series 

00 

(41) y = E c n x n 

in eq. (40) and obtain conditions on the coefficient c n . In general c , C\ 
turn out to be arbitrary constants and the other coefficients are expressible 
in terms of these two with the aid of a recursion formula. The series eq. 

(41) converges and represents a solution of eq. (40) for |a;| < a, provided 
p{x) and q (x) are analytic functions of the complex variable a; for \x\ < a; 
see Chap. 7. 

If p(x) and q(x) are not both analytic at x = 0, but xp{x) and x 2 q(x) are 
analytic at x = 0, then x = is called a regular singular point of Eq. (40). 
In this case a solution is sought in the form 

00 

(42) y = S c n x n+k , 

n=0 

where the c's and k are constants. If one substitutes in eq. (40) and equates 
to .zero the coefficients of the various powers of x, one obtains a set of 
equations relating the c's and k. The coefficient of the lowest power of x 
gives rise to a quadratic equation in k, called the indicial equation. Four 
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cases arise, depending on whether the two roots of this equation do not 
differ by an integer, are equal, or differ by an integer. In two of these 
cases, the solution consists partly of power series multiplied by log x. 
For further details and examples of the important applications, one is re- 
ferred to Chap. 7, Sect. 9. (See also Refs. 3, 9, 11.) 

7. SIMULTANEOUS LINEAR DIFFERENTIAL EQUATIONS 

Attention will be restricted to systems of first order equations: 

dyi n 

(43) -r" = S a ij( x )Vj + Qi(%), i= 1, • • • , n. 
dx i=1 

A variety of other systems can be reduced to this form by appropriate sub- 
stitutions. It will be assumed that the a^(x) and Qi(x), •••, Q n (x) are 
continuous for a < x < b. 

By a solution of eqs. (43) is meant an n-tuple of functions 

(44) y 1 =fi(x), •••, y n = f n {x), a<x<b 

which together satisfy eqs. (43) identically. The general solution of eqs. 
(43) can be shown to have form 

(45) yi = crfufr) + c 2 f 2 i(x) -\ \- c n f ni (x) + gi(x); 

here i = 1, • • • , n, the c's are arbitrary constants; yi = gi(x) (i = 1, • • • , n) 
defines one solution of eqs. (53) ; for eachj the functions fa(x) (i = 1, • • • , n) 
define a solution of the related homogeneous system 

dyi A 

(46) — = 2^ a>%m %z= *> ' • •» n > 
ax j = i 

and these solutions of eqs. (46) are linearly independent; that is, 

i>ifu(x) + b 2 f 2 i(x) -\ [- b n f ni {x) =0, i = 1, • • • , n 

implies 6i = 0, • • • , b n = 0. 

Homogeneous Systems with Constant Coefficients. Consider the 
homogeneous system eq. (46) in which the a^- are constants. Particular 
solutions are obtained by setting 

(47) ?/i = ai e* x , -•-,y n = a n e^. 
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Equations (47) define a solution provided X satisfies the characteristic equa- 
tion 

an — X ctj2 

a 22 — X 



(48) 



021 



ai n 

a 2n 



«nl 



a n2 



= 0. 



If X satisfies eq. (48), then values of a ly 
that 



■ , a n not all can be found such 



(49) 



2 ifla - ^ij)<*j = (i = 1, • • •, n), 



3=1 



where 8{j = if i y± j, = 1 if i = j. With these values of the a i} eq. (47) 
defines a solution of the system (46). If the characteristic equation has 
distinct real roots Xi, • • • , X n , then one obtains n linearly independent solu- 
tions in this manner, so that the general solution is obtained in form (45), 
with the gi(x) replaced by 0. If X is a repeated root of multiplicity k, one 
replaces eq. (47) by 



(50) 



Xz 



Vi = Pi(z)e , '",y n = Vn{x)e Kx , 



where Pi(x), • • •, p n (%) are polynomials of degree at most k — 1; one can 
obtain k linearly independent solutions in this form, which can be used to 
build the general solution. The procedure is easily modified to take care 
of complex roots (Refs. 1, 3). 

The methods described are more easily formulated in terms of matrices 
(Chap. 3, and Ref. 3). 

Nonhomogeneous Systems. If the general solution of the related 
homogeneous system is known : 



(51) 



Vi = Cifii{x) H 1- Cnfni{x) (l = 1, • • •, 7l) , 



then one can obtain a particular solution by the method of variation of 
parameters. One replaces ci, •••, c n in eq. (51) by unknown functions 
vi(x), • • •, v n {x). Substitution in eqs. (43) leads to the equations 



dvi 



(52) l^Jji— = W*W, * = i, • • •, n, 

j = i ax 

which can be solved for the functions dvj/dx; integration then yields vi(x), 
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• ' ' , v n (x) and the solution 

(53) iji = vi(x)f u (x) -\ h v n (x)f ni (x), i = 1 



> ) 



For systems with constant coefficients one can also employ operational 
methods (Chaps. 8, 9). 

8. NUMERICAL METHODS 

Method of Picard. Let a system 

dy% 

(54) — = Fi(x, yi, •••, y n ), i=l,--,n, 
ax 

be given and let a solution 

(55) 2/1 = fi(x), ••-, y n = f n (x), a<x<b, 
be sought with initial values 

(56) fi(x ) = fci, • • -yfnM = k n , a < Xq < b. 

One forms successive approximations to the desired solution by the for- 
mulas : 

f\*{x) = fa, '■•,fn l (x) s k n , 

fi 2 (x) = I Fi(t, fa, • • -, k n ) dt + fa, i = 1, • • -, n, 

J XQ 

and in general 

jr + \x) = f Fsj^it), -■■,fn m {t)) dt + fa, 

Jxo 

f or i = 1, • • • , n. Under appropriate hypotheses (see Ref . 3) the sequences 
fi m (x) (i = 1, ■ • • , n) converge to the desired solution (55) as m — > <x>. 
An equation of order n 

(57) y w = F(x,y, ■■',y (n ~ 1) ) 
can be replaced by the system 

/MV dyi dy 2 dy n _ x dy n 

(58) — = y 2 , — = 2/3, •••>— — = Vn t — = F(x, y h • ••,#„) 

ax ax ax ax 

and then treated in the same way. 

Step-by-Step Integration. An approximation to the solution (55) 
can be obtained by replacing (54) by the corresponding incremental equa- 
tions 

(59) Ay { = Fi(x, y u • • •, y n ) Ax. 
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The values of ij), • • • , y n at x = x are the given constants k\, • • •, k n . 
The values at Xq + Ax are ki -f- Ay\, • • • , k n + Ay n , where the Ayt are 
computed from eq. (59) with F{ evaluated at xo, k\, • • •, k n . Proceeding 
in this manner, step by step, one obtains values of yi, • • -, y n at discrete 
values xo, xq + Ax, • • • . The increments Ax can be varied or kept con- 
stant. Under appropriate hypotheses, the "solution" thus computed con- 
verges to the desired solution as the increments Ax approach 0. (See 
Ref. 3.) 

Other Methods. A variety of more refined numerical methods have 
been developed, in many cases well suited to analysis on digital computers. 
For details see Chap. 14. 

9. GRAPHICAL METHODS— PHASE PLANE ANALYSIS 

The discussion will be limited to the first order equation 

(60) y' = f(x, y) 
and to the system 

dec du 

(61) - = F(x, y), -j- = G(x, y). 
at at 

Elimination of t between the two eqs. (61) leads to an equation of form 
(60) and, indeed, eqs. (61) should be thought of as a parametric form of 
eq. (60) ; t can be interpreted as time. It should be remarked that certain 
second order equations can be reduced to these forms. For example an 
equation 

d 2 x / dx\ 

(62) — =f(x,-) 



dt 2 \ dt/ 



is equivalent to the pair 



dx dy 

(63) - = y, -=}{x,y). 
dt dt 

Method of Isoclines. For the eq. (60) the isoclines are the loci of 
constant slope; that is, the loci 

(64) f(x, y) = m. 

In general, eq. (64) defines a family of curves in the xy-pl&ne. If one 
draws a series of short line segments, all with slope m, and each with its 
center on the curve, then one has obtained a series of tangent lines to the 
unknown solutions. If the process is repeated for many different values of 
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m, one obtains a dense set of such tangent lines. From these tangent lines, 
the solutions are usually readily sketched. An example is suggested in Fig. 
1. 




Fig. 1. Solution of y' = x — y by isoclines. 



Singular Points. The form (60) has certain disadvantages which can 
be avoided by using an appropriate equivalent form (61). In particular, 
f(x, y) may have discontinuities, while the equivalent form (61) need have 
none. For example, the equation y' = (x 2 — y 2 )/xy can be replaced by 



(65) 



dx 



dy 
dt 



= x 2 — y 2 : 



the new system has no discontinuities. Equations (65) can be thought of 
as defining a vector field in the xy-plane. Instead of drawing tangent lines 
as above, one can draw many vectors at scattered points in the xy-plane. 
These suggest the solution curves in the same way as do the tangent lines. 
The vector field fails to define a direction only where both dx/dt and 
dy/dt are 0. For eqs. (65) this holds only at the origin. For the general 
system (61) this holds where both equations 

(66) F(x, y) = 0, G{x, y) = 

are satisfied. The solutions of eqs. (66) are called singular points. 

A graphical analysis near the singular points is generally difficult to com- 
plete with the aid of isoclines alone. It is possible to obtain a qualitative 
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picture of the solutions near the singular point by expanding F(x, y), G(x, y) 
in power series and neglecting all but the linear terms. If the singular 




Fig. 2. Solution near focus type singular point. 

point is at (0, 0), one thus replaces eqs. (61) by the approximating linear 
system 



(67) 



dx dy 

— = a x x + biy, — = a 2 x + b 2 y. 

dt dt 



The appearance of the solutions of eqs. (61) near the singular point in 
typical cases is suggested in Figs. 2, 3, 4, 5. The arrows on the curves indi- 




Fig. 3. Solution near saddle-point type singular point. 
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cate the direction of increasing time t. The four cases illustrated correspond 
to four cases for the roots of the characteristic equation 



(68) 



o'i ~~ X fri 

a 2 b 2 — X 



= 0. 



In Fig. 2 the roots are — a ± /3z, with a > 0, /3 > 0; in Fig. 3 the roots 
are X 1; X 2 with Xi < < X 2 ; in Fig. 4 the roots are ±/3i, /3 > 0; in Fig. 5 




Fig. 4. Solution near center type singular point. 

the roots are Xi, X 2 with Xi < X 2 < 0. The solutions of the system (61) 
will have the same appearance near (0, 0) as the solution of eqs. (67), 
except in borderline cases; of the four cases illustrated, only that of Fig. 4 
is of borderline type. For a full discussion, see Ref . 2. 




Fig. 5. Solution near node type singular point. 
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Limit Cycles. Of much importance for applications are the solutions 
represented by closed curves in the xy-p\ane. These are termed limit cycles. 
For the parametric eqs. (61) such a solution is represented by equations 
x — p(0> V = <?(0> where p and q have a common period T. A typical 
solution family containing a limit cycle C is illustrated in Fig. 6. The 




Fig. G. Limit cycle. 

cycle C is stable in this case ; that is, all solutions starting near C approach 
C as time t increases. In many cases simple properties of the isoclines allow 
one to conclude existence of limit cycles in particular regions. A theorem 
of Bendixson states, that a region in which F x -f- G y > can contain no 
limit cycle of eqs. (61) (Refs. 2, 3). 

Phase Plane. For the motion of a particle of mass m on a line, classical 
mechanics gives an equation of the form 



(69) 



d 2 x ( dx\ 

m — = F [ t, x, — , 
dt 2 \ dt/ 



When F is independent of t, the substitution v = dx/dt leads to an equation 

dv 



(70) 



n\v — = F(x, v) 
' dx 



which can be analyzed as above. The pair (x, v) represents a phase of the 
mechanical system and the rrw-plane is termed the phase plane. Second 
order equations arising in other contexts can be treated similarly and the 
term phase is used for the pair (x, v) or (x, y) regardless of the physical 
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significance of the variables. An especially simple graphical discussion can 
be given for the conservative case of eq. (69) : 

(71) m-^ = F(x). 

dr 

See Ref. 2. 

10. PARTIAL DIFFERENTIAL EQUATIONS 

This section presents a brief discussion of partial differential equations 
of second order. Some further information is given in Chap. 6. (See 
Refs. 4, 10.) 

Classification. Consider an equation 

b 2 U b 2 U b 2 U bu bu 

(72) A — - + 2B HC— + £> \- E f- Fu + G = 

bx fdxdy by dx by 

where u is an unknown function of x and y and the coefficients A , - • • , G 
are given functions of x and y (perhaps constants). The eq. (72) is termed 

elliptic if B 2 - AC < 0, 

-parabolic if B 2 — AC = 0, 

hyperbolic if B 2 — AC > 0. 

The three types are illustrated by the 

• b 2 u b 2 u 

Laplace equation: — - -\ = 0, 

bx 2 by 2 

\bu b 2 u 

heat equation: k — - = 0, 

bt bx 2 

b 2 u b 2 u 

wave equation: — - — k — - = 0. 
bt 2 bx 2 

Attention will be restricted to the three special types. 

Dirichlet Problem. One seeks a solution u(x, y) of the Laplace equa- 
tion in an open region D, with given boundary values on the boundary of 
D. This problem can be treated by conformal mapping (Chap. 10, Sect. 5). 

Heat Equation. A typical problem is the following. One seeks a solu- 
tion u(x, t) of the heat equation u t — k 2 u xx = for t > 0, < x < 1, with 
given initial values <j>(x) = u(x, 0) and boundary values w(0, t) = 0, 
u{\, t) = 0. To obtain a solution one can employ the method of separa- 
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Hon of variables. One seeks solutions of the differential equation and 
boundary conditions of form 

(73) u = f{x)g(t). 

From the differential equation one finds that one must have 

™ - ^ = 

0(0 m 

Hence g'/g must be a constant X, and/"// must equal \/k 2 : 

(74) g'(t) - \g(t) = 0, k 2 f"(x) - \f(x) = 0. 
From the boundary conditions at x = and x = 1 one finds that 

(75) /(0) = /(l) = 0. 

From eqs. (74) and (75) one concludes that f(x) and X must have the form 

(76) f(x) = b sinnTrx, X = —k 2 n 2 ir 2 , n = 1, 2, 

From eqs. (74) g(t) has form const. -e xt . Hence particular solutions of form 
(73) have been found: 

(77) u = e~ k n v l sin mrx, n = 1, 2, 

Each linear combination of the functions (77) is also a solution of both the 
heat equation and the boundary conditions at x = and x = 1. Accord- 
ingly, each convergent series 

CO 

(78) u = Y, b n e- k ' Wt sm mrx 

n=l 

also represents a solution. By proper choice of the constants b n the initial 
values can be satisfied. One requires that 

oo i 

(79) 4>(x) = 52 b n sin mrx. 

Thus the b n are determined from the expansion of 4>(x) in its Fourier sine 
series (Chap. 8, Sect. 8). With the b n so chosen, eq. (78) represents the 
desired solution of the given problem. 

Wave Equation. One seeks a solution u{x, t) of the wave equation 
u t t — k 2 u xx = for < x < it, t > with given initial values u(x, 0) = 
4>{x) and initial velocities u t (x, 0) = ip{x) and given boundary values 
u(0, t) = u(ir, t) = 0. This is the problem of the vibrating string. The 
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method of separation of variables can be used as above and one obtains 
the solution in the form of a series 

(80) u =V23 , 'sin nx[a n sin knt + /3 n cos knt], 

n=\ 

where a n and /3 n are determined from the expansions : 

oo oo 

(81) <t>{x) = ^2 (3 n sin nx, \p(x) = ^2 nka n sin nx. 

n=l n=l 

Relaxation Methods. One can obtain an approximation to the solu- 
tion of a partial differential equation by replacing it by a corresponding 
difference equation. The method has been especially successful for the 
Dirichlet problem, which is discussed here. The differential equation 
u X x + u yy = is replaced by the equation 

(82) u{x -\- h, y) + u{x, y + h) + u(x — h, y) 

+ u(x, y — h) — 4u(x, y) = 0. 

If the given region is the square 0^x^1,0^2/^1, one chooses h = 
\/n for some positive integer n and requires eq. (82) to hold at the lattice 
'points {k\h, k 2 h), < &i < n, < k 2 < n. The values of u on the bound- 
ary (x = or 1, y = or 1) are given, and eq. (82) becomes a system of 
simultaneous linear equations for the unknowns u{k\h, k 2 h). These can be 
solved by the relaxation method. One chooses an initial set of values for 
the unknowns, then obtains a next approximation by replacing u(x, y) by 

(83) \[u(x -\-h,y)-\r u(x, y + h) + u(x - h, y) + u(x, y — h)] 

at each lattice point. Repetition of the process generates a sequence 
Un(%, y) which can be shown to converge to the solution of eq. (82). As 
h — » 0, the solution of eq. (82) can be shown to converge to the desired 
solution of the Dirichlet problem (Ref. 10). 
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1. DEFINITIONS AND MAIN PROBLEMS 

A linear integral equation of first kind is an equation of form 

(1) f K(s,t)x(t)dt=f(s); 

J a 

f(s) and K(s, t) are considered to be given, and a function x(t) satisfying 
eq. (1) is called a solution of the integral equation. 

Fredholm Integral Equation. This is the linear integral equation of 
second kind and has the form 

(2) x(s) - X f K(s, t)x(t) dt = f(s). 

J a 

6-01 
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Here K(s, t) and f(s) are given real functions, and X is a given real con- 
stant; a solution of the integral equation is a function x(s) satisfying eq. 

(2) for a ^ s ^ b. 

Volterra Integral Equation. If in eq. (2) the upper limit b is replaced 
by the variable s, the resulting equation 

(3) x(s) - X f K(s, t)x(t) dt = f(s) 

J a 

is called a Volterra integral equation. Equation (3) can be considered as a 
special case of eq. (2) ; namely, the case for which K(s, t) = f or t ^ s. 

The preceding definitions relate to integral equations for functions of 
one real variable. There are analogous definitions for functions of two or 
more real variables. It is also of importance to allow x, K, f to take on 
complex values and to allow X to be complex. For simplicity the results 
will be formulated for functions of one variable; essentially no change is 
required to extend the results to functions of several variables. Only func- 
tions with real values will be considered here. The discussion will further- 
more be restricted to the integral equation of second kind; for the equation 
of first kind, see Ref . 10, Chap. 2. 

Remark. The equations defining Laplace and Fourier transforms can 
be regarded as integral equations of first kind. Solving the equations is 
equivalent to finding the inverse transforms. See Chaps. 8, 9. 

The function K(s, t) in eq. (2) is called the kernel of the integral equation. 
The eq. (2) is said to be homogeneous if /(s) = 0; otherwise it is nonhomo- 
geneous. The homogeneous equation 

(4) x(s) - X J K(s, t)x{t) = 

J a 

obtained from eq. (2) by replacing /(s) by is called the homogeneous equa- 
tion associated with eq. (2). 

A number X such that eq. (4) has a solution x = <£(s) not identically is 
called a characteristic value or eigenvalue of eq. (4) or of the kernel K(s, t) ; 
the solution <£(s) is called an eigenfunction associated with X. For each 
eigenvalue X there may be several associated eigenf unctions. From the 
definition it follows that cannot be an eigenvalue. 

The eigenvalue problem associated with eq. (4) is the determination of 
whether, for a given kernel, eigenvalues exist, what they are, and what 
the corresponding eigenfunctions are. 

The expansion problem associated with eq. (3) is the determination of 
the possibility of expanding every function g(s) of a given class in an in- 
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finite series: 

CO 

0( s ) = Z)c a a (s), 

a=l 

where the 4> a (s) are eigenfunctions. 

The solvability problem associated with eq. (2) is the determination of 
whether eq. (2) has a solution x(s) and whether the solution is unique. 

2. RELATION TO BOUNDARY VALUE PROBLEMS 

The problems described in Sect. 1 arise naturally in the analysis of bound- 
ary value problems associated with partial differential equations. 
A Typical Example is presented. The equation is the wave equation 

(5) u u — V 2 w = 0, u = u(x, y, z, t), 

where V 2 is the Laplacian operator: 

d 2 u d 2 u d 2 u 

(6) v " = ^ + v + ^- 

A finite domain D, with smooth boundary B, is given in x, y, z space; a 
function g(x, y, z) is given in D. One seeks a function u satisfying eq. (5) 
for t ^ and for (x, y, z) in D and satisfying the boundary conditions 

(7) u(x, y, z, t) = 0, (x, y, z) on B, t ^ 0; 

(8) u{x, y, z, 0) = g(x, y, z), (x, y, z) in D; 

(9) u t (x, y, z, 0) = 0, (x, y, z) in D. 

The classical attack on this problem is to "separate" the time and space 
variables; that is, to set 

(10) u(x,y,z,t) =S(x,y,z)T(t). 
Then one is led to the boundary value problems: 

(11) V 2 £ + XS = 0, S = for (x, y, z)- on B. 

(12) T"(t) + \T(t) = 0, T'(0) = 0. 

If S and T satisfy eqs. (11), (12) for some constant X, then u = ST satis- 
fies eqs. (5) and both (7) and (9), but not necessarily eq. (8). 

Integral Equation. The problem (11) can be replaced by an integral 
equation by the following reasoning. It is shown in the theory of partial 
differential equations (Ref . 5) that there exists a uniquely determined func- 
tion K(s, a) of the two points 

s: (x, y, z), <r: (£, rj, f) 
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in D, the so-called Green' 's function, with the following properties: K has 
continuous second partial derivatives as long as s ^ ir; K(s, a) = for s 
on B, or in D; if 4>{a) has continuous first partial derivatives in D and Di is 
an arbitrary subdomain of D (with smooth boundary), then 

(13) V 2 JJJk(x, y,z, £, v , £)<£(£, v, f) ft dr, tff = -<Kx, y, z) 

for (x, y, z) in Z>i. Identifying <j> with — X$ and Di with D, one sees that 
the boundary value problem (11) is equivalent to the homogeneous inte- 
gral equation in these variables: 

(14) S(x, y, z) -\ffJK(x, y, z, £, v , f)S(£, r,, f) d$ dr, d{ = 0. 



X) 



Solution. Let eq. (14) have a sequence of eigenfunctions $«(#, y, z) 
associated with the positive eigenvalues X a (a = 1, 2, •••)• Then # a 
satisfies eq. (11) with X = X a ; for X = X a eq. (12) has the solution T a = 
cos v X a t, so that u = S a T a satisfies eqs. (5), (7), (9). To satisfy eq. (8), 
one notes that each series 

00 

(15) u = 23 c a S a {x, y, z)T a (t) = 2c a S a cos V X a t 

a = l 

also satisfies eqs. (5), (7), (9), if the c's are constants and the series satisfies 
appropriate convergence conditions. The condition (8) now becomes 

00 

(16) g(x, y,z) = J2 c aS a (x, y,z); 

a=l 

thus one is led to the expansion problem. If g can be expanded as in eq. 
(8), then (15) defines a solution of the given problem. 

Suppose that the on the right-hand side of eq. (5) is replaced by 
F (x, y, z, t)) this corresponds to an external force. If F Q (x, y, z, t) = 
F(x, y, z)T(t), where T{t) satisfies eq. (12) for some X = X , then the sub- 
stitution of eq. (10) leads to the nonhomogeneous integral equation 

(17) Six, y, z) - X jJJK(x, y, z, £, v , f)SU, v, f) d$ dr, d{ = f(x, y, z), 

D 

where 

(18) /(*, y, z) = JfJK(x, y, z, £, ,, fl^tt, 77, f) df dr, #. 
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3. GENERAL THEOREMS 

In what follows K(s, t) will be assumed continuous for a ^ s ^ b, 
a ^ t ^ b. Such a continuity condition is not always satisfied, e.g., for 
the Green's function of Sect. 2. See Ref. 5 (pp. 543 ff.) and Ref. 6 (pp. 
355 ff.) for reduction of the discontinuous case to the continuous case. 

Definitions. The following definitions relate to functions of t defined 
and continuous for a ^ t ^ b. If x, y are two such functions, their scalar 
'product is 

(19) (x, y) = f x{t)y(t) dt. 

J a 

The norm || x || of x = x(t) is defined as (x, x) y *. Functions x\, • • • , x n are 
linearly independent if 

(20) c lXl (t) +--+c n x n (t) =0, 

with constant c\, • • •, c n , implies c\ = 0, • • •, c n = 0; if the functions are 
not linearly independent, they are termed linearly dependent. An infinite 
system of functions 

(21) 01, <t>2, 

is called linearly independent if $i, - - • , 4>k are linearly independent for 
every k. 

Two functions x, y are said to be orthogonal if (x, y) = 0. The system 

(21) is orthogonal if (0 a , 4>p) = for a ?± /3. A system of orthogonal func- 
tions none of which is identically zero is necessarily linearly independent. 
The system (21) is called orthonormal if it is orthogonal and normalized, 
that is, 

|| <j> a || = 1 for all a. 

If the system (21) is linearly independent, it can be orthogonalized and 
normalized; that is, an orthonormal system {\f/ a } can be found such that, 
for every n, \[/ n is a linear combination of 0i, • • •, 4> n and 4> n is a linear 
combination olypi, • • • , ^ n - For details, see Ref. 4, p. 50. 

The Schwarz Inequality states that for every x, y, \ (x, y)\^ \\ x \\ • \\ y \\, 
with equality if and only if x, y are linearly dependent. 

The Bessel Inequality states that, if {(f> a } is an orthonormal system, 
then for every x 

(22) El(z,*J| 2 ^ Ml 2 . 

a=l 
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Now consider three related integral equations : 

(23) 0(s) - X f K(s, 00(0 dt = /(«), 

•'a 

(24) 0(s) - X J X(s, £)<K0 d« = 0, 

J a 

(25) ^(s) - X f X(£, s)^(0 d< = 0. 

J a 

Equation (24) is the homogeneous equation related to (23); eq. (25) is 
called the adjoint or transposed equation of (24). 

Theorem 1. 7/ X is an eigenvalue of eq. (24), then X is an eigenvalue of 
eq. (25) . There are at most a finite number k of linearly independent eigen- 
f unctions of eq. (24) associated with eigenvalue X; this maximal number k is 
the same for eq. (25). 

The number k is called the multiplicity of the eigenvalue X. 

Theorem 2. Equation (23) has a solution if and only if f is orthogonal to 
all solutions of the adjoint eq. (25). 

Conclusions Based on Theorems 1 and 2. Let X not be an eigenvalue 
of K(s, t). Then X is also not an eigenvalue of Kit, s) ; that is, ^ = is the 
only solution of eq. (25). Hence (/, \p) = for all solutions yp of eq. (25), 
and eq. (23) has a solution for arbitrary f. For eachf, the solution is unique; 
for the difference of two solutions is a solution of eq. (24), hence = 0. 

Let X be an eigenvalue of K(s, t). Then eq. (25) is satisfied for at least 
one \p not identically zero and eq. (23) is not satisfied for somef, in particular 
for / = yp. (In the problem of Sect. 2 this case arises if the frequency X of 
the time factor T of F (x, y, z, t) is an eigenvalue of the homogeneous eq. 
(14) ; this is the case of resonance.) 

One is thus led to the following alternative of Fredholm: either (i) the 
nonhomogeneous eq. (23) has a solution for arbitrary / or (ii) the homo- 
geneous eq. (24) has at least one (not identically vanishing) solution. 
Case (i) can also be characterized by the statement: eq. (23) has at most 
one solution for each /; for the uniqueness implies existence of a solution. 

4. THEOREMS ON EIGENVALUES 

The kernel K(s, t) is said to be symmetric if K(s, t) = Kit, s). This case 
occurs in many applications ; for example in the problem of Sect. 2. 

Theorem 3. A symmetric kernel has at least one and at most a countable 
infinity of eigenvalues. Eigenfunctions corresponding to distinct eigenvalues 
are orthogonal. The eigenvalues can be numbered to form a sequence {X a }, in 
which each eigenvalue is repeated as many times as its multiplicity, and such 
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that | Xi | ^ | X2 1 ^ • • • I if there are infinitely many eigenvalues, then | X a | — > 00 
as a — » co. An eigenfunction 4> a can be assigned to each X a in such a fashion 
that the sequence {4> a } is orthonormal and every eigenfunction cf> is a linear 
combination of a finite number of the $ a 's. 

The sequence {(/>«} is called a full system of eigenfunctions of the kernel. 

Remark. While restricting K(s, t) to be real, one can consider complex 
eigenvalues X and eigenfunctions x(t) = Xi(t) + ix 2 (t). Some kernels have 
only complex eigenvalues; some kernels have no eigenvalues at all. A sym- 
metric kernel has only real eigenvalues. 

5. THE EXPANSION THEOREM AND SOME OF ITS CONSEQUENCES 

Theorem 4. Let {<f> a } be a full system of eigenfunctions for the symmetric 
kernel K{s, t). Then in order that a function g{s) can be expanded in a uni- 
formly convergent series: 

(26) g(s) = £ c«0«(s), 

a 

where 

(27) c a = (g, </>„), 

it is sufficient that g(s) can be written in the form 

(28) g{s) = f K(s, t)G(t) dt, 

J a 

where G(t) is continuous. 

In many applications the form (28) for the function g(s) to be expanded 
arises in a natural way. For example, the function (18) is of this form. 

The coefficients (27) can be written in a different form which is often 
useful. From eq. (28) and from the facts that 4> a satisfies eq. (24) with 
X = X a and that K is symmetric, one deduces the expression 

,™x (^> ^«) 

(29) c a = — 

X a 

As a, first application of the expansion theorem, let X be a number which 
is not an eigenvalue, and seek to expand the solution x = <f>(s) of eq. (23) 
in terms of the eigenfunctions. To do this, note that by eq. (23) x — f is 
of form (28) with G = Xx. By Theorem 4 and eq. (29) one deduces the 
expansion 

(30) x{s) - f{s) = E — fo 0«)0«(*). 

If this relation is multiplied by 4>$(s) and integrated from a to b, one ob- 
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tains a linear equation for (re, <pp). Solving this equation and substituting 
the result in eq. (30) gives the desired formula 

(3D *(*)=/(«)+ A E^4 *«(«)• 

a A a — A 

(The series is meaningless if X is one of the \ a , unless (/, # a ) = 0; this is 
in agreement with Theorem 2 of Sect. 3.) 

A second application concerns the "quadratic form" 

nb n b 

(32) I{x, x} = I I K(s,t)x(s)x(t)dtds, 

J a J a 

whose importance will become clear in the next section. If one applies the 
expansion theorem to the integral of K(s, t)x(t), one obtains the formula: 

k a 2 

(33) I{x, x) = Z)t~ i K = (x, <f> a ). 

a A a 

The transition from eq. (32) to (33) is the analogue of choosing coordinates 
which represent a conic section in its "principal axis" form. 

6. VARIATIONAL INTERPRETATION OF THE EIGENVALUE PROBLEM 

In this section the hypotheses and notations are the same as those of 
Sect. 5. It is convenient to denote the positive X a 's by 

(34) < Pl ^ p 2 ^ p 3 ^ ■ ■ • 
and the negative ones by 

(35) > -toi ^ -n 2 ^ -n 3 ^ • • • . 

There may be no p's or no w's; as remarked in Sect. 1, is not an eigen- 
value. Equation (33) now becomes 

(36) I{x,x] = Z— -£-, 

j Pj j Uj 

where kj = (re, fy), lj = (re, xi) and \f/j is the eigenfunction associated with 
Vh Xj the eigenfunction associated with nj. From eq. (36) and Bessel's 
inequality (22) one now concludes: 

Theorem 5. // there are positive eigenvalues of the symmetric kernel 
K(s, t), then 

II x || 2 

(37) I { XtX )£lLJL i 

Pl 
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where p\ is the smallest positive eigenvalue. The maximum of I{x, x\ for x 
within the class of x having norm 1 is attained when x = \{/i and equals l/p\. 
If there is a positive eigenvalue p n , then the maximum of I{x, x] within the 
class of x for which 

(38) 11*11 = 1, (*,*;) =0 (j= 1, -.n-l) 

is attained when x = \f/ n and equals l/p n - 

If K(s, t) is replaced by —K(s, t), one obtains a characterization of the 
negative eigenvalues and corresponding eigenfunctions. 

The characterization of eigenvalues in Theorem 5 is recursive; that is, in 
order to characterize p n and \p n one has to know ^i, • • •, rf/ n —i- A direct 
characterization is obtainable as follows. Let M{yi, • • • , y n —\] denote the 
least upper bound of I{x, x\ among all x such that 

(39) (*,y/)=0 (j= 1, ..-,n-l). 

It can be shown that, among all choices of y± = y\{t), • • •, y n -i = y n -i(t), 
M has its smallest value, namely l/p n , when y± = yp\, • • •, y n —i = ^n-i- 
See Ref. 4, p. 132. 

Rayleigh-Ritz Quotient. This is the quotient 



(40) Q{x} = I{x, x} + f\ f K(s, t) x(t) dt 



ds. 



Assume that there are at most a finite number of negative eigenvalues 
and assume all the eigenvalues are numbered so that Xi ^ X2 ^ X3 ^ • •• • . 
From eq. (33) one finds 

(4i) /Kx} = E(~) 2 x«ax 1 z(rT- 

a \A a / a \A a / 

From the expansion theorem of Sect. 5, with G{t) = x(t), one deduces that 

r b f r h "I 2 /k \ 2 

(42) K(s,t)x(t)dt d8=Z(r)' 

J a L«/ a J a \A a / 

From eqs. (40), (41), (42) one thus obtains the inequality 

(43) Q{x) ^ Xi. 

Furthermore one can show that Q{x) takes on its minimum X x when 
x = 0i. Thus the smallest eigenvalue and associated eigenf unction are ob- 
tainable by minimizing Q{x}. This is the basis of a very effective computa- 
tional procedure. 
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The quotient Q{x} can be written in another way, more familiar in the 
theory of differential equations. One sets 

(44) u(s) = \ K(s, t)x{t) dt, 

J a 

so that 

/%b nb 

(45) Q{x) = I u(t)x(t)dt+l u 2 dt. 

J a J a 

The analogous definition, and integration by parts, for the problem of Sect. 
2 leads to the expression 

1 1 1 (Ux 2 + u y 2 + u 2 ) dx dy dz 

(46) Q{x\ = — , 

I I I u 2 dxdy dz 

D 

where u is the solution of the problem 

(47) V 2 u = —x in D, u = on B. 

7. APPROXIMATION METHODS 

The first four methods to be described are devices for replacing the inte- 
gral equation by a system of linear algebraic equations. 

Approximation of Integrals. Let a subdivision of the interval 
a ^ t ^ b be given: 

a = ti < t 2 < • • • < t n < t n+ i = b 

and let 5 = max (£y +1 — t } ). Then for continuous h(t), the difference 



„b n 

•^O 7 = 1 



can be made as small as desired, in absolute value, by making 8 sufficiently 
small. Hence one can take the sum as approximation to the integral. If 
this is done for the Fredholm eq. (2), one obtains the approximating 
equation 

n 

(48) x(s) - X S K(s, tj)(t i+1 - tj) = f(s). 
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If one now writes 

(49) x{U) = x it K{U, tj)(t j+1 - tj) = a ijt f(U) = bi 
f or i = 1, • • •, n, then at s = U eq. (48) becomes 

n 

(50) Xi — X 2Z a H x i = °i (i = 1, • • •, n). 

3 = 1 

This is a system of linear equations for x\, • • •, £ n . A solution can be re- 
garded as giving the values of the desired x(s) at t\, • • • , t n ; one can inter- 
polate linearly between these points to obtain an approximation to x(s). 
The first proof by Fredholm of the main theorems of Sect. 3 was based on 
eq. (50) and subsequent passage to the limit (n — > <», 5 — > 0). 

For numerical purposes the procedure may be improved by using better 
approximations for the integral such as those given by the trapezoidal rule, 
Simpson's rule or Gauss's quadrature (Ref. 9, Chap. 7). Each of these 
methods replaces the integral by a sum 2h(tj)Aj with properly chosen 
abscissas /_,- and "weights" Aj. For more details and also the question of 
convergence, see Ref. 1 (pp. 105 ff.), Ref. 3 (pp. 437 ff.), Ref. 9 (p. 455). 

Method of Degenerate Kernels. A kernel A (s, t) is called degenerate 
if it can be written as a finite sum of products of a function of s by a func- 
tion of t; that is, if it is of the form 

n 

(51) A(s,t) = E^-(sW). 

Every continuous kernel K(s, t) can be approximated by a continuous degen- 
erate kernel A (s, t) ; that is, for every e > there exists a continuous A (s, t) 
such that \K(s, t) — A(s, t) | < e for a ^ s ^ b, a ^ t S h. One therefore 
obtains an approximate solution of eq. (2) by replacing K by A. For the 
question of convergence one is referred to Ref. 4 (pp. 118 ff.), Ref. 1 
(Abschnitt IV), and Ref. 3 (p. 464). 
If K is replaced by A, the Fredholm eq. (2) is replaced by the equation 

(52) x(s) - X E M s ) I Bj(t)x(t) dt = /(s), 

j=l J a 

whose solution is found by solving a system of linear equations. To see 
this, one multiplies eq. (52) by B{(s) and integrates with respect to s from 
a to b. With the notations 

p.b ^,6 p>b 

J x(t)Bj(t) dt = x h I Aj(t)Bi{t) dt = ay, I f(t)Bj(t) dt = f j} 

J a •* a J a 
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one obtains the system 

n 

(53) X{ — X 2 a H x i = fi '(* = !> * - •> w )- 

y=i 

It can be verified that if x\, • • • , x n is a solution of this linear system, then 

n 

(54) x(s) = f(s) + X E XjMs) 

y=i 

is a solution of eq. (52) and, conversely, every solution of eq. (52) is ob- 
tained in this way. 

The Ritz-Galerkin Method. This is a method for finding approxima- 
tions to the eigenvalues and eigenfunctions of the homogeneous eq. (3) 
with symmetric kernel. Let {v a \ be an orthonormal system. Such a sys- 
tem is called complete (in the class of continuous functions on the interval 
a =^ s ^ b) if for every continuous function x(s) the sums 

n „b 

(55) 2 3,i>*(s), Xi = I Xi(t)vi(t) dt 

1=1 Ja 

converge "in the mean" to x(s); that is, if 

pb n 

lim I [x(s) — ^ ^A( s )] 2 ds = 0. 

Example. The functions 

v\ = (2x) - ^, ^2m+i = tt~ 1//2 cos ms, y 2w = ir~ y2 sin ms, (w = 1, 2, • • •) 

form a complete orthonormal system for — -k ^ s ^ 7r; see Chap. 8, Sect. 
8. 

Now let 0i (s) be a normalized eigenfunction of eq. (3) corresponding to 
the smallest positive eigenvalue Xi. (If there are no positive eigenvalues, 
one follows a similar procedure starting with the negative eigenvalue of 
smallest absolute value.) One now seeks an approximation 4> to <f>\ of form 

n 

(56) 0=2 c^i(s), 

i=i 

where {v a } is a complete orthonormal system. In order to determine the 
C{, note (Theorem 5, Sect. 6) that 1/Xi is the maximum of I{x, x) when 
|| a; || = 1, and that this maximum is reached for x = <fo. Restricting at- 
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tention now to functions of form (56), one finds 

n n 

(57) / {<£, 0} = £ ^b ali C a Cp, 

a=\ 0=1 

with 

(58) b a p = I I K(s, t)v a (s)vf}(t) ds dt. 

J a " a 

The condition || 4> || = 1 becomes 

n 

(59) Z c,- 2 = 1. 

i=i 

Maximizing the quadratic form (57) with side condition (59) can be ana- 
lyzed by the method of Lagrange multipliers (Ref. 4). One obtains the 
equations 

n 

(60) c { - X J2 bijCj = (i = 1, • • •, n) 

which, together with eq. (59), determine the C{ and X. In particular, X is 
a root of the algebraic equation obtained by setting the determinant of 
eq. (60) equal to zero. If Xi* is the smallest positive root of this equation, 
then Xi* is an approximation to Xi and Xi* ^ Xi; for X = Xi* eqs. (59) and 

(60) determine C\, • • • , c n and, by eq. (56), a desired approximation $ of 
the eigenf unction fa. 

Method of Enskog. The method will be discussed for the Fredholm 
eq. (2) with symmetric kernel, with X not an eigenvalue. (For less restric- 
tive assumptions, see Ref. 7, p. 109.) It is based on a complete linearly 
independent system Vi, v 2 , • • • with the additional property that the func- 
tions 

(61) y n (s) = v n (s) - X f K(s, t)v n (t) dt 

J a 

are orthonormal and complete. Such a system can be constructed as fol- 
lows: Let Wi, w 2 , • • • be a complete linearly independent system (e.g., the 
system of sines and cosines given above). One then defines 

(62) z n (s) = w n (s) - X f K(s, t)w n (t) dt. 

J a 

It can be proved that the z n are likewise linearly independent and com- 
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plete. From the z n one constructs an equivalent orthonormal system 
2/i, 2/ 2j • • • (Sect. 3), so that relations 

n 

(63) y n (s) = J2 c nmZ m (s) 
hold, with constant c nm . One now defines: 

n 

(64) v n (s) = ^c nm w m (s). 

m=l 

It follows from eq. (62) that eq. (61) holds. Moreover, the system {v n } is 
a complete linearly independent system. 

Having a system {v n } of the properties indicated, one can find an ap- 
proximate solution x(s) of the Fredholm eq. (2) of form 

n 

z(s) = X) c iVi( s ), c i = (P, y { ). 

Multiply eq. (2) by V{(s) and integrate with respect to s from a to b, to 
obtain the relations: 

pb pb „b 

(f, Vi) — I x(s)vi(s) ds — X I I K(s, t)x(t)vi(s) dt ds 

J a J a Ja 

(*b „b 

= I x(s)[ Vi (s) - X I K(t, s)Vi(t) dt] ds. 

•J a J a 

Because of the symmetry of the kernel, the expression in brackets is yi(s). 
Hence 

,b 



(/, v { ) = I x(s)yi(s) ds = 

J a. 



Iteration is the basis of the following methods: 

Successive Approximations. The Fredholm equation can be written 
in the form 

(65) x(s) = /(s) + X f K(s, t)x(t) dt. 

J a 

This form suggests defining successive approximations x {l) (s) to the solu- 
tion x(s) as follows: 

(66) x {0 \s) = f(s), x (<+1) (s> = /(.s) + X f K(s, t)xS { \t) dt, 
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where i = 0, 1, 2, • • •. One can prove by induction that 

(67) x M (s) = f(s) + 2>* f K (i \s, 0/(0 dt, 
where the so-called iterated kernels are defined by the relations 

(68) K (1) (s, = K(s, t), K^ i+1) (s, t) = f K(s, u)K™(u, t) du, 

J a 

for i = 1, 2, • • •. It can be proved that 

(69) x{s) = lim x (n) (s) = f(s) + £ V I ^ (i) («, ^ 

w — » °° 1=1 J a 

exists if |X| is less than [(& — a) Max K(s, 0] —1 > the series, known as 
Neumann's series, converges uniformly for a ^ s ^ b. The function x(s) 
defined by eqs. (69) is the solution of (65) for X restricted as stated. For 
the Volterra eq. (3) the Neumann series converges for all X and the solu- 
tion is valid for all X. 

The Schwarz Constants. Write I{x, x, K] for the quadratic form 
I{x, x} defined by eq. (32) to express more clearly the dependence on K. 
The Schwarz constants are then defined as follows: 

(70) a = (x, x), a t = I{x, x, K w ), (i = 1, 2, • • •), 

where the K M are defined by eq. (68). These constants (which obviously 
depend on the choice of the function x) are important for the theory as 
well as for estimating eigenvalues. Note the following facts, supposing al- 
ways that K (s, is symmetric : 

If P is an arbitrary real number, subject only to the restriction that 

(71) a i+i - Pa i+2 9± 0, 

and 

- - a i+l P 



(72) Q = 



Ot+l — O-i^P 



then the interval with end points P, Q contains at least one eigenvalue pro- 
vided that at least one of the. following assumptions is satisfied: (a) i is 
even, (b) K is a positive definite kernel, that is, I{x, x) > unless x = 0. 
(For a proof, see Ref. 1, p. 30.) The quotients Q are termed Temple 
quotients. Setting P = in eq. (72) leads to consideration of the quotients 
Qi = ai_\/ai. It can be shown that the sequence | Qzi-i I is monotone 
nonincreasing and converges to | Xi | , where Xi is the eigenvalue of smallest 
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absolute value. (For further applications of the Schwarz constants, see 
Refs. 1, 3, 9.) 

Method of Steepest Descent. The basis of this method is the fact 
that x is a solution of the Fredholm eq. (2) with symmetric kernel if and 
only if x minimizes the expression 

pb nb 

(73) F\x\ = §[(z, x) - X I I K(s, t)x{s)x(t) dsdt] - (/, x). 

J a J a 

Let Xq be a first approximation to x. One seeks a better approximation 
X\ = xq + h, and tries to choose h so that in going from x to Xi the value 
of F descends as rapidly as possible. With the notation 

(74) L[x] = x - X f K(s, t)x(t) dt, 

J a 

one finds that 

(75) F{x + h} = F{x \ + (L[x ] -f,h)+ $(L[h], h). 

Now if F were a function of a finite number of real variables, the analogue 
of the second term on the right side of eq. (75) would be the scalar product 
of grad F with h. One therefore defines here 

(76) g[x] = L[x]-f 

as the gradient of F. This suggests that, as in the case of a function of a 
finite number of real variables, the direction of steepest descent is given by 
the negative gradient; this can be proved to be true. One therefore sets 
h =■ —ag[x ], where a is a real constant to be determined. Replace h by 
— ag[x ] in eq. (75); then F[x + h] becomes a function of the real variable 
a. Now determine a by minimizing this function by the ordinary methods 
of calculus. The result for the desired next approximation X\ = x + h is 

(77) xi = x g[xo\- 

(L[g[x ]\, g[x ]) 

If one repeats the procedure starting with x\ instead of x , one obtains a 
new approximation x 2 ; continuing thus, one obtains a sequence X\, x 2 , • • • , 
x n , • • • . If the kernel K is symmetric and positive definite and | X | is less 
than |X a | for every eigenvalue X a , then x n converges in the mean to the 
solution x of the Fredholm eq. (2). For proofs and details, see Ref. 8 (pp. 
103 and 136) . The method can also be applied to finding eigenvalues (Ref. 
8, p. 142). 
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1. FUNCTIONS OF A COMPLEX VARIABLE 

Complex Numbers. Throughout Chap. 7, z = x + iy and w = u + iv 
denote complex numbers; i is the imaginary unit, i 2 = —1; x, ,y, u, v are 
arbitrary real numbers; x is the real part of z, y the imaginary part of z: 

(1) x = Re (x + iy), y = Im (x + iy). 

The complex numbers z can be represented geometrically by the points 
(x, y) of an x?/-plane (or z-plane), as in Fig. 1. The polar coordinates (r, 6) 
of z are termed respectively the modulus (or absolute value) of z and argu- 
ment (or amplitude) of 2: 

(2) r =\z\= mod z; = arg z = amp z; z = r(cos 5 + * sin 6). 

7-01 
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The conjugate of z = x + iy is: 



(3) 



z = x — %y. 



Algebraic properties of complex numbers are discussed in Chap. 2. In 
general, complex numbers are combined as are real numbers, with the rela- 
tion i 2 = — 1 used to simplify the results. Addition is the same as vector 



Zl + Z 2 




Fig. 1. The complex z-plane. 

addition (Fig. 1). Multiplication of Z\ by z 2 yields a number z\-z 2 whose 
modulus is | Z\ \ • \ z 2 | and whose argument is arg Z\ + arg z 2 . 
Useful Rules. 



(4) 



Zl + z 2 = Zi + z 2 , 
2 + 2 = 2Re (z), 



Zi-Z 2 = Zi-Z 2 , 

z — z = 2i Im (z), 



z-z = Izl 2 , 



|Zl +Z2| ^|2l| + |22|, I 22 - Zl| ^| |22| "Nil \, 

z n = [r(cos 6 + i sin 0)] n = r n (cos nd + t sin w0), n = ±1, ±2, 



Complex Functions. By a function of the complex variable z will be 
meant an assignment of a value w to each z of a certain set D in the z-plane 
(see Chap. 1, Sects. 1 and 3); one then writes: 



(5) 



w = /(z). 



(Some formulas will assign several values of w to each z in D. One then 
speaks of a "multiple-valued function.") The set D is generally an open 
region (e.g., interior of a circle); see Chap. 1, Sect. 8. From the equation 
u -f- iv = f(x + iy) one deduces two equations of the form 



(6) 



u - u(x, y), 



v = v(x, y) {{x, y) in D), 
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and conversely a pair (6) of functions of two variables determines a com- 
plex function (5) of z. 

Limits and continuity for complex functions are defined as for real func- 
tions. The phrase "z approaches z " is interpreted to mean: \z — z \ — » 0, 
or that the distance from z to z becomes arbitrarily small. The basic 
theorems on sums, products, quotients hold without change from the real 
case. Continuity of w = f(z) is equivalent to continuity of both u(x, y) 
and v(x, y) in (6). 

Each complex function w = f(z) can be interpreted as a mapping (Chap. 
10) of the set D into a set E in the w-plane. If f(z) is continuous, then as 
z traces a curve in the z-plane, w traces a curve in the w-plane. 

Derivatives of complex functions are defined as for real functions: 

d f(z + Az) - f(z) 

(7) -/(*) = /'(*) = lim - / ' , 

dz as -> o Az 

and the formal rules of differentiation carry over. Higher derivatives 
f"(z), • • • are defined similarly. 

Definite integrals of complex functions are defined as line integrals: 

(8) f /(z) dz = f(u + iv) (dx + i dy) 

= J u dx — v dy + i I v dx + u dy. 

Here C is a continuous path of finite length from Zi to z 2 . Again the 
formal rules carry over. 

Examples of Complex Functions are the following: 

polynomials: w = a z n + • • • + a n —\Z + a n , 

a z n H \- a n 

rational functions: w = , 

b z m +--.+ bJ 

exponential junction: w = e z = e z (cos y + i sin y) = exp z, 

logarithm: w = log z = log \z\ + i arg z (z 9^ 0), 

power function: w = z a = exp (a log z), 

e iz - e~ iz e iz + e~ iz 

trigonometric functions: sin z = ; , cos z = 



(9) 



2i 2 

e z — e~ z e z + e~ z 
hyperbolic functions: sinh z = , cosh z = 
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inverse trigonometric functions: 

sin -1 z = - log (iz ± VI — s 2 ), cos -1 2 = - log (2 ± iVl — z 2 ) 



The logarithm is a multiple-valued function and can be made single- 
valued (so that continuity can be discussed) by properly restricting z and 
the choice of arg 2. The principal value is: 

(10) log 2 = log \z\ + id, (r > 0, -7T < ^ tt), 

a function continuous except for 6 = it. 

If a is a rational number (e.g., %), z a has a finite number of values. For 
example, z y * = -\/z has two values : 

^A — e V2 log 2 _ gj^dog r+i arg 2) _ -y/^ g (^)i arg 2 
(ID 

= V r ( cos — f- * sin - J or V r ( cos f — h tt ) + * sin ( — \- t 1 j , 

if 8 is one choice of arg z. 

Identities satisfied by the exponential function, logarithm, and trigo- 
nometric functions: 

e z x +z 2 = e^i.g^ e *i-z 2 = e *i ^_ e ^ ( e *)n = e nz ( n = ±lj ±2 , • • •), 

(12) log (2i -2 2 ) = log 2! + log 2 2 , log 2 W = n log 2, 

sin (21 -f- 2 2 ) = sin 21 cos z 2 + cos 21 sin 2 2 , sin 2 2 -f- cos 2 2 = 1, 

In the case of the logarithm, the identities are true only for proper choice 
of value of each logarithm concerned. The rules for differentiation also 
carry over: 

d 

(13) — (2 n ) = nz n -\ 

dz 

2. ANALYTIC FUNCTIONS. HARMONIC FUNCTIONS 

The function w = f{z) is said to be analytic (regular, holomorphic) in 
an open region D if it has a derivative f'(z) in D. The function f(z) is 
analytic in D if and only if u = Re (J(z)) and v = Im (f(z)) have continuous 



d 


d 


— e s = e", 


— sin 2 = cos 2, 


dz 


dz 
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first partial derivatives in D and the Cauchy-Riemann equations hold in D : 

du dv du dv 

dx dy dy dx 

Furthermore, if f(z) is analytic, 

du dv dv dv 

(15) /'(«) = — + i — = — + i- = • • -. 

dx dx dy dx 

If /(z) is analytic in D, the derivatives of all orders of /, u, v exist and 
are continuous in D. From eq. (14) one deduces that 

d 2 u d 2 u d 2 v d 2 v 

dx z dy 1 dx 2 dy 1 

that is, u and v are harmonic functions. Relations (14) are described by 
the statement: "u and v form a pair of conjugate harmonic functions." 
One says "v is conjugate to u," but should note that u is conjugate to —v. 
In polar coordinates (14) and (15) become 



(17) 



du 1 dv 1 du 


dv 


— = = — 


— 


dr r dd r dd 


dr 


.. (du dv\ 


• 



(18) 



All the functions (9) are analytic, provided the logarithms are restricted 
so as to be continuous and division by zero is excluded. A function ana- 
lytic for all z is called an entire function or an integral function; examples 
are polynomials and e z . 

A function cannot be analytic merely at a single point or merely along 
a curve. The definition requires always that analyticity holds in an open 
region. The phrases "analytic at z " or "analytic along curve C" are 
understood to mean "analytic in an open region containing z " or "ana- 
lytic in an open region containing curve C." If / is analytic in an open 
region D, then the values w = f{z) form an open region in the w-plane. 

3. INTEGRAL THEOREMS 

The open region D is termed simply connected if every simple closed path 
C in D (Fig. 2) has its interior in D. If D is not simply connected it is 
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multiply connected; for example, the region between two concentric circles 
is multiply connected; it is doubly connected, because its boundary is formed 
of two pieces or "components." 




Fig. 2. Simply connected region. 

All paths in the following line integrals are assumed to be "rectifiable," 
i.e., to have finite length. 

Cauchy Integral Theorem. If f(z) is analytic in a simply connected 
open region D, then 



J> /(: 



z) dz = 

on every simple closed path C in D or, equivalently, I f(z) dz is independent 
of path in D. 

Morera's Theorem (converse of Cauchy theorem). If f(z) is continuous 
in the open region D and 



fy(z) dz = 



on every simple closed path C in D, then f(z) is analytic in D. 

An indefinite integral of f(z) is a function F(z) whose derivative isf(z). 
If f{z) is continuous in D and has an indefinite integral F(z), then 

(19) f 2 f(z)dz = F(z 2 )-F(z 1 ); 

in particular, the integral is independent of path, so that f(z) must be ana- 
lytic; since F'(z) = f(z), F(z) must also be analytic. If f{z) is a given ana- 
lytic function in D, then existence of an indefinite integral of f(z) can be 
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proved, provided D is simply connected. In particular, 

(20) F(z) = f f(z) dz (20 in D) 

has meaning, since the integral is independent of path, and F'(z) = f(z), 
so that F is an indefinite integral. 

Cauchy Integral Formulas. Let f(z) be analytic in D. Let C be a 
simple closed path in D and having its interior in D. Let z be interior to C 
(Fig. 2). Then 



/(zo) = — <P dz, f'(z ) = — Ay -dz, 

2iri cJ z — Zn 2tti r J (z — 



m 

z ' 2iri c y(z — z ) 2 

(21) 



n\ C f(z) 
f M (z )=— A) \ n ..- dz, 

2ffl nJ (Z — Zn) n 



2iri c J(z - z ) n+1 

At the heart of this theorem is the special case f(z) = 1 : 

r dz r dz 

2^i=A) ; = <h- -, n = 2,3, •••. 

c Jz - z cJ(z-z ) n 

Cauchy's theorem and integral formulas can be extended to multiply 
connected domains. Let D be a domain bounded by curves C\, C 2 , • • • , C& 




Fig. 3. Multiply connected region. 

as in Fig. 3. Let f(z) be analytic in a somewhat larger region, including 
all of D and its boundary. Then 

(22) <£/(«) dz + £f(z) dz + • • • + £f(z) dz = 0; 

that is, the integral of f(z) around the complete boundary B of D is zero, 
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provided one integrates on the boundary in the direction which keeps the 
region D "on the left": 

(23) ff(z)dz = 0. 

Under the same conditions, if z is in D, 

1 C f( z ) , , n\ r f(z) 

(24) /(zo)=— -^— dz, / (n) (*o)= — T- J ^^+l dz - 

2-M nJ Z — Zn 2tI nJ (Z — Zn) ^ 



Cauchy Inequalities. Under the hijpotheses staled above for eqs. (21), 
let \f(z) | = M on C and let C be a circle with center z and radius R. Then 

Mn\ 

(25) |/(») (2o) |g (n = 0, 1,2, •••). 

ti 

Liouville Theorem. // f(z) is analytic for all finite z and \ f(z) \ ^ M , 
where M is a constant, for all z, then f(z) is identically constant. 

Maximum Principle. Let f(z) be analytic in the open region D. If 
\f(z) | has a weak relative maximum at a point z of D (that is, if \f(z) \ 
= l/( z o) I f or z sufficiently close to z ), then f(z) is identically constant. 

For proofs of these theorems see Refs. 2, 3, 8. 

4. POWER SERIES. LAURENT SERIES 

Infinite series whose terms are complex numbers are defined as for real 
numbers and, in general, the theory of convergence is the same. In par- 

ticular, a series ^3 °n of complex numbers is termed absolutely convergent 

if the series of real numbers 2 1 b n \ converges. Absolute convergence implies 
convergence. 

Power Series. A power series in z has the form 

00 

(26) 2 c n (z - z ) n , 

where z is fixed. Each such series has a radius of convergence p, ^ p 
^ +=o. If p = 0, the series converges only for z = z . Otherwise, the 
series converges (in fact, absolutely) for \z — z \ < p, i.e., inside the circle 
of convergence (whose radius p may be infinite). Outside this circle, for 
\z — z | > p, the series diverges. On the circle: \z — z \ = p, the series 
may converge at some points and diverge at others. The radius can be 
evaluated by the formulas: 

1 

p = lim 



(27) p = lim 



J n+l 



V 
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provided the limit exists, and in any case by the formula 

1 

(28) p = lim n/ - — - . 

n— >m V | C n | 

where lim denotes the lower limit. 

Let the power series (26) have radius of convergence p > 0, so that its 
sum is a well-defined function f(z) inside the circle of convergence. One 
can then prove that the series converges uniformly to f(z) in each circle 
1 2 ■— z 1 = p' < p, so that /(z) is continuous. Furthermore, the differen- 
tiated series 2nc n (z — z ) n_1 converges uniformly in each circle \z — z \ 
^ p' < p. From this it follows that the differentiated series converges to 
/'(z) and that/'(z) is continuous. Hence /(z) is itself analytic for \z — z \ 
< p. Every power series defines an analytic function inside its circle of con- 
vergence. In general, all derivatives of /(z) can be evaluated by repeated 
differentiation of the series. One hence concludes that 

(29) c n = — ; 

w! 

that is, the power series is the Taylor series of f{z). From this it follows 
that equality of the sum of two power series : 

00 00 

X) c n (z - z ) n = X) C n (z - z ) n , |z - Z | < p, 

n=0 n=0 

implies equality of corresponding coefficients: 

c n = C n (n = 0, 1,2, ..-). 

Now let f{z) be given as an analytic function in an open region D of 
arbitrary shape and let z be a point of D. With z as center one can then 
construct a circle of maximum radius r having its interior in D (Fig. 4). 




Fig. 4. Taylor series expansion. 
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Within this circle f(z) can be represented by a power series, its Taylor series 
about Zq : 

™f n \z Q \ 

(30) /(z) = E — — (z - z ) n , \z-z \<r ; 

n=o n\ 

the series may have a radius of convergence p larger than r . From this 
theorem one deduces the following expansions : 

e z — 2_) — ' all z ; sin z = z - + ■••, all z ; 

n=0 n! 3! 

z 2 

cos 2 = 1 \- • • • , all z ; 

2! 

(31) log (1 + 2) = 2 -^- + ^--..., |z|<l; 

= l + z + z 2 +-.-, |z|< 1; 



1 - 2 

fc(& - 1) o 
(l+z) & = l + fcz + Z 2 +---, \z\< 1. 

Li I 

Laurent Series. A series of form 

00 b 



.ti (2 " 2 ) n 



is reducible by the substitution 2' = l/(z — z ) to the form of an ordinary 

power series and accordingly converges for \z'\ < p, i.e., for |z — z | > pi 

1 
= - . If now a series ]T) a n {z — z ) n converges for \z — z \ < p 2 and 

p n=0 

Pi < P2> then the sum 

^ + X a n(z - ZQ)" 



n=l (2 — Zo) n n =0 

has meaning for pi < | z — z 1 < P2, that is, in a certain annular region D 
(Fig. 5). Here pi may be and p 2 may be +°°. Let the sum be /(z), so 
that /(z) is analytic in D. If one writes b n — a_ n (n = 1, 2, • • •), then 

one has 

— 1 00 00 

(32) f(z) = 2 o»(* - *o) n + E «n(2 - 2 ) n = E a n (z - z ) n . 

n= — 00 n=0 ra= — 00 
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This series is termed the Laurent expansion of f(z) . 
be shown to be uniquely determined as follows: 



The coefficients can 



(33) 



2iri nJ(z — 



f(z) dz 



2iri c y(z — z ) 



n+\ 



n = 0, ±1, ±2, 



where C is any path about the ring, as in Fig. 5. 

If /(z) is an arbitrary function analytic in a ring domain D, then one 
can compute the coefficients a n by eq. (33) and form the Laurent series, 




Fig. 5. Laurent expansion. 

which will then converge to /(z) in D. In practice there are easier ways of 
obtaining the coefficients. One way is to write /(z) as the sum of two func- 
tions./^),/^), the first analytic for \z — z \ < p 2 , the second analytic for 
\z — z \ > pi and approaching as \z\ — > oo. Under the substitution 
£• = \/{z — z Q ), fi(z) becomes a function of f analytic for |f | < 1/pi, so 

00 

that A (2) = E&nf B or 



(34) 



A(a) = E 



&n 



=1 (2 - ZqY 



I 2 ~ 2o|> PL 



For/ 2 (z) one has a Taylor series about z . Addition of the two series pro- 
vides the desired Laurent series. For example, if f(z) = l/(z — l)(z — 2) 
and D is the ring 1 < \z\ < 2, then one can choose /i(z) = —1/(2 — 1), 
f 2 (z) = 1/(2 - 2). 

5. ZEROS. SINGULARITIES. RESIDUES. ARGUMENT PRINCIPLE 

Zeros. Let f(z) be analytic in domain D and let f(z ) = 0. Then z is 
called a 2ero of /(z) . If f(z) is not identically zero, then each zero has a 
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definite order (or multiplicity) n, a positive integer, and/(z) = (z — z ) n g(z) 
where g is analytic in D and g(z ) 7^ 0. The order n is the smallest value 
of k such that / (ft) (z ) ^ 0. If /(z) is not identically zero, then each zero 
of /(z) is isolated; that is, for each zero z one can choose a circular region 
\z — z I < a containing no other zero. 

Singularities. Let /(z) be not identically zero and have a zero of order 
n at Zq. Then h(z) = l//(z) is analytic in some circular region | z — z \ < a 
except at the center Zq. By definition, h{z) has a pole of order n at z . One 
can write: h(z) = (z — z ) _n p(z), where p(z) is analytic for |z — z | < a 
and p(z ) ?^ 0. Since /(z ) = 0, l/i(zH — > co as z — > z . One conven- 
tionally assigns the value <» to h(z) at z - 

In general, let /(z) be analytic in a punctured disk: < \z — z \ < a, 
but not at z . Then /(z) is said to have an isolated singularity at z . One 
can form a Laurent expansion of /(z) in the ring domain pi = < | z — z 1 
< a = p 2 . Three cases can then arise. 

I. No negative powers in the Laurent series. Then 

00 

/(Z) = X) a n(z ~ Z ) n , 
n=0 

so that /(z) can be treated as a function analytic for \z — z$\ < a without 
exception. The singularity is termed removable. The new value of /(z) at 
zq is a = lim f(z). 

z—>zo 

II. A finite number of negative powers in the Laurent series. Here, for 
proper choice of JV, 

a_N tt_i 

/(z) = — -\ 1 \- a + ai(z -...zq) H 

(z - z ) iV z - z - 



(35) 



flf(«) 



(z - Zo)" 



Sf(z ) = a_ N 9^ 0. 



Hence /(z) has a poZe of order N at Zo- 

III. Infinitely many negative powers in the Laurent series. In this case 
/(z) is said to have an essential singularity at Zo. 

By a theorem of Riemann, the three cases can be distinguished as fol- 
lows: I. |/(z) I is bounded for < | z — z 1 < b for some 6. II. |/(z) | — > 00 
as z — » z . III. Neither |/(z) | nor l/|/(z) | is bounded in each punctured 
disk <\z — z | < b. In Case III, by a theorem of Weierstrass and 
Casorati, /(z) comes arbitrarily close to every complex number in every 
neighborhood of z . 
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If /(z) is analytic for \z\> R, then /(z) is considered to have an isolated 
singularity at z = ». A Laurent expansion is available, with p\ = R and 
p 2 = co. The classification is similar to the above, with "negative" replaced 
by "positive." Also the type of singularity of f(z) at co i s the same as that 
of/(l/z) atz = 0. 

A function analytic for all finite z except for poles is termed a meromorphic 
function. 

Residues. The residue of f(z) at an isolated singularity z is defined as 

(36) Res[/(z),z ] = — ff(z)dz, 

where C is a circle | z — z 1 = c, enclosing no singularity other than z , and 
the integration is in the counterclockwise direction. The residue of f(z) 
at z = oo, denoted by Res [/(z), co], is defined by the same integral, where 
C is a circle \z\ = c outside of which /(z) has no singularity other than co 
and where the integration is in the clockwise direction. If z is finite, 

(37) Res [/(z), z ] = a_ b 

where a_i is the coefficient of (z — z ) _1 in the Laurent expansion about 
z . If z is co, 

(38) Res [/(z), oo] = -a_i, 

where a_i is the coefficient of z~ l in the Laurent expansion of f(z) for 
\z\> R. 

The Cauchy Residue Theorem asserts that, iff(z) is analytic in an open 
region containing the path C, then 

(39) u)/(z) dz = 2iri-(sum of residues of f{z) inside C), 

provided f(z) is analytic inside C except for a finite number of isolated singu- 
larities. Similarly, 

(40) Sf(z) dz = 2ivi- 

c (sum of residues of f(z) outside C, including Res [f(z), co]), 

provided f{z) is analytic outside C except for a finite number of isolated singu- 
larities. Hence, if f{z) is analytic for all z, except for a finite number of singu- 
larities, the sum of all residues of f(z), including that at <x>, is 0. 
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Calculation of residues may be simplified by the following rules : 

1. At a pole z of first order, 

(41) Res [f(z), z ] = lim (z - z )f(z). 

2. At a pole z of order N (N = 2,3, • • •), 

(42) Res [/(*), z ] = lim "— ^, 

«-«„ (N - 1) ! 

where g(z) = (z - z ) N f(z). 

3. Let 

A(z) 



(43) f(z) = 



B(z) 



where A(z) and 5(z) are analytic at z . If A(z ) ^ and J3(z) has a zero 
of first order at z , then 

4(zo) 

(44) . Res [/(*), zo) = ~^- 

B'(z ) 

If A(z ) ^ and ^(2) has a zero of second order at z , then 

QA'(z )B"(z ) - 2A(z )B"'(zo) 

(45) Res [/(z), z ] = — — - 1 — • 

' 3[£"(z )] 2 

If A (z ) 7^ and B{z) has a zero of third order at z , then 

(46) Res[/(z),z ] 

_ 12QA"B'" 2 - 60A'B'"B iv - 12AB'"B V + \5AB lv 2 
~ 405'" 3 ' 

where all quantities are evaluated at z . If A(z) has a first order zero at 
z and B(z) a second order zero, then 

2A'(z ) 



(47) Res [f(z), z ] = 



B"(z ) 



Argument Principle. Let /(z) be analytic in an open region D containing 
the simple closed path C; let f(z) have at most a finite number of singularities 
inside C, all of which are poles, and let f(z) ^ on C. Then 



1 f/'(z) 

(48) v- wH dz 

2ri C J f(z) 



= number of zeros of / inside C — number of poles of / inside C, 
where zeros and poles are counted according to multiplicity. 
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The left-hand side of eq. (48) is termed the logarithmic residue of f(z) on 
C. It can be written as 



— tydlogf. 



2iri 

As z traces C, w = f(z) traces a path C w in the w-plane. The integral 

I d log f(z) equals i times the total change in the argument of w as the 

path C w is traced. Hence it equals 2wi times the "winding number" of 
C w about w = 0, i.e., the number of times that C w effectively winds about 
w = in the positive direction. 

The Fundamental Theorem of Algebra (see Chap. 2). From the 
argument principle one deduces that every polynomial in z of degree N has 
precisely N zeros in the complex plane. 

Rouche's Theorem may also be deduced: if both /i(z) and f 2 (z) are 
analytic in a simply connected open region containing the simple closed 
path C and \fi(z) — f 2 (z) | < |/ 2 (2) | on C, then /i (z) and/ 2 (z) have the same 
number of zeros inside C. 

Evaluation of Definite Integrals by Residues. A great variety of 
definite integrals can be evaluated with the aid of residues. For example, 
if R(u, v) is a rational function of u and v, then 



r 2,r r A - i z + 1\ dz 

(49) I ft(sin 9, cos 6) dd = <h R ( . ) — 

J J \ 2iz 2z / iz 

l*l = i 

and the integral on the right can be computed by residues. Also, in general 

(50) I f(x) dx = 2iri { sum of residues of f{z) in the half-plane y > } , 

provided f(z) is analytic for y ^ except for a finite number of points in 
V > 0> I /( x ) ^ exists and 

«/ — 00 



lim f /(^e i( 



i9\ p„i'0 j/i = 



)Ee l " d^ = 0. 



The last condition is satisfied if f(z) is rational and has a zero of order 
greater than 1 at «>, or if f(z) = e miz g(z) where m > 0, ^(2) is rational, and 
#(z) has a zero at 00. For further applications one is referred to Chap. VI 
of Ref. 12. 
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6. ANALYTIC CONTINUATION 



Let /i(z) be analytic in the open region Di, f 2 (z) in D 2 . If D 2 and D\ 
have a common part and/i(z) = f 2 (z) in that common part, then f 2 (z) is 
said to be a direct analytic continuation of /i(z) from Di to D 2 . Given 
/i(z), Z>i, Z>2, -the function / 2 (2) may or may not exist; however, if it does 
exist, there can be only one such function (uniqueness of analytic continua- 
tion) . 

Let Z>i, D 2 , • • •, D n be regions such that each has a common part with 
the next and let f 3 -(z) be analytic in Dj (j — 1, • • •, w). If //(z) = /y+i(z) 
(j = 1, • • •, n — 1) in the common part of Dy, -Dy+i, then one says that 
/i(z) has been continued analytically from Di to D n via D 2 , • • -, D n -i and 
calls f n (z) an (indirect) analytic continuation of /i(z). Given /i(z) and the 
regions Di, • • • , D n , there is at most one analytic continuation of /i(z) to 
D n via D 2 , • • •, Z) n _i. There may exist other continuations of fi(z) to D n 
via other chains of regions. 

Given a function f(z) analytic in region D, one can form all possible 
continuations of f(z) to other regions. The totality of such continuations 
is said to form an analytic function in the broad sense (Weierstrassian ana- 
lytic function) . In this sense log z, s/z, sin -1 z can each be considered as 
one analytic function. The importance of the concept is illustrated by the 
fact that every identity satisfied by f(z) will be satisfied by all its analytic 
continuations. The term "identity" includes linear differential equations 
with polynomial coefficients. 

Example of Analytic Continuation. The functions 

00 z n 

/.(z) = £-fr UK 2, 

A (z + 1)" 

n=0 <J 

are analytic continuations of each other. Indeed, both are power series 
expansions of / 3 (z) = 1/(2 — z) and have the same sum for \z\ < 2. Also 
f 2 (z) can be regarded as the Taylor series of fi(z) about z = — 1. This 
series happens to converge outside of | z \ < 2 and hence provides an ana- 
lytic continuation. 

Analytic Continuation from Reals. Let f\(z) be defined only for 
y = 0, a < x < b, i.e., only when z is real and between a and b. Let f 2 (z) 
be analytic in an open region D which includes the interval of definition 
of fi(z). Iif 2 (z) = fi(z) on this interval, then/ 2 (z) is said to be an analytic 
continuation of /i(z) from reals. Again continuation, if possible, is unique. 
Examples. e z as a continuation of e x , sin z as a continuation of sin x, 
log z as a continuation of log x. 
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7. RIEMANN SURFACES 

The function w = z y% can be considered as an analytic function in the 
broad sense; that is, it is formed of several functions which are analytic 
continuations of each other. The resulting totality has the defect that it 
is two-valued : for each z, there are two possible values (except for z = 0) . 
To remedy this defect one regards w = z Vi as a function defined not in the 
z-plane, but on a Riemann surface over the z-plane. In this case, the Rie- 
mann surface can be constructed as follows. One takes two copies of the 
z-plane, calling them Sheet I and Sheet II. Each sheet is considered as 
cut open along a branch line, the positive real axis. Sheet II is placed 
directly over Sheet I, with axes in the same position, and then the two 
sheets are attached by joining upper edge of the cut line of each sheet to 
the lower edge of the cut line of the other, as suggested in Fig. 6. Un- 



A z-plane 




Fig. 6. Riemann surface of z y% . 



Fig. 7. Branch line for w = z y K 



fortunately this cannot be carried out in space. For each point in the 
z-plane, one has then two points in the Riemann surface, one in each sheet. 
As one traces a path about z = in the z-plane, one can describe a corre- 
sponding path in the Riemann surface by assigning a sheet to each posi- 
tion; no change of sheet can be made except when crossing the branch 
line, and a change of sheet must be made at such a crossing (Fig. 7). A 
closed path in the z-plane will not in general lead to a closed path on the 
Riemann surface. A path which closes up after two encirclements of the 
origin will be closed on the Riemann surface. The origin itself appears as 
a point common to the two sheets and is termed a branch point. 

On the Riemann surface just constructed one can now define \/z as a 
single-valued function as follows: -\fz = -\/r e l612 , < 6 < 2ir, on Sheet I; 
z = \/r e 1012 , 2ir < 6 < Al-k, on Sheet II. Above the branch line continuity 
determines the proper value to be assigned. 

The procedure described can be generalized to 



w = Vz, 



w 



= V(z - l)(z -2)(z -3) 
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and to all algebraic functions. In general n sheets will be required and 
several branch lines and branch points. The surface for 



w 



V(z - l)(z -2)(z -3) 



is suggested in Fig. 8. 

The procedure can be extended to nonalgebraic functions, but in gen- 
eral infinitely many sheets are required. An important case is log z, for 




Fig. 8. Riemann surface of 
w = [(z - l)(z - 2)(z - 3)] M . 



M 1 
z-plane 








-II -I 





I II 




-II -I 





I II 



Fig. 9. Riemann surface of 
log z. 



which sheets 0, ±1, ±11, • • • are needed, as in Fig. 9. In this case z = 
is a logarithmic branch point and is not regarded as a point of the Riemann 
surface. 

8. ELLIPTIC FUNCTIONS 

Let f(z) be a meromorphic function (analytic except for poles) ; /(z) is 
said to have period co, co ^ 0, if f(z + co) = f(z) for all z; f(z) is called an 
elliptic or doubly periodic function if / is not constant and has periods wi, co 2 
and if oi\/co 2 is not real. It then follows that nicoi + n 2 co 2 are also periods, 
for every choice of the integers n 1} n 2 . For proper choice of wi, co 2 these 
are all the periods of / and it will always be assumed that a>i, co 2 are so 
chosen. The numbers 12 = n\wi + n 2 w 2 form the vertices of a paving of 
the plane by parallelograms, any one of which can be chosen as a period 
parallelogram of f(z) ; it is convenient to exclude the points on a pair of 
adjacent sides from each period parallelogram. It can be proved that/(z) 
has a finite number N of poles (counted according to multiplicity) in a 
period parallelogram; N is the order of f(z) as an elliptic function; N is 
always at least 2. In general, /(z) — a has N zeros in the parallelo- 
gram. 

Jacobian Elliptic Functions. Examples of elliptic functions are pro- 
vided by the functions 

sn z, en z, dn z 
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of Jacobi. These can be denned as follows. For fixed k, < k < 1, let 

(51) F(w) =J„ vi-wi E(w) =1 Vl -* 2 - 2 ""; 

Z* 7 and £ are called the elliptic integrals of Legendre of first and second type, 
respectively. The equation z = F(w) can be solved for w to yield a mul- 
tiple-valued function w of z, w = am 2 ; finally 

sn z = sin (am z), en z = cos (am z), 

(52) 



dnz = Vl - /c 2 



sn z. 



Despite the multiple-valued operations, the functions sn z, en z, dn z are 
defined as single-valued functions, analytic except for poles and with power 
series expansions about z = 0: 

z 3 

snz = z - (1 + k 2 ) — -| , 

3! 

z 2 z 4 

(53) cnz = l--+(l + 4fc 2 )- + ..., 

k 2 z 2 

dn z = 1 1 . 

2! 

All these functions depend on the parameter k. They have periods coi, co 2 
as follows; for sn z, w\ = 4tK and co 2 = 2iK'; for en z, «i = 4K, u 2 = 
2K + 2iK'; for dn z, Ul = 2Zf, w 2 = 4^'; here 



T/2 d« /7T 

Vcos 2 < + k 2 sin 2 « \2 



(54) K'=f Q — ._ 2 /.\, a „._ 2 / ^ = ^(^ 



Hence sn z, en z, dn z are elliptic functions. Tables of the functions F, E } 
sn z, en z, dn z are available; see, for example, G. W. and R. M. Spenceley: 
Smithsonian Elliptic Function Tables (The Smithsonian Institution, Wash- 
ington, 1947), and L. M. Milne-Thompson: Jacobian Elliptic Function 
Tables (Dover, New York, 1950). 

It can be shown that the most general elliptic function is expressible 
simply in terms of sn z and en z. Furthermore, a large class of integrals, 
called elliptic integrals, can be expressed in terms of the functions F and E 
and the integral of third kind: 

r w dt 

(55) t(w) = 

Jo 



(1 + a 2 sin 2 0Vl - k 2 sin 2 t 
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which depends on k and the real parameter a. The general elliptic integral 
has form I R(z, V P(z) ) dz, where R{z, w) is a rational function of z and 

w, and P is a polynomial of degree three or four with distinct roots. 

In the theory of elliptic functions a number of additional functions ap- 
pear, of which the following are important definitions and relations: 

The Weierstrass (P-Function. 



(56) 



<P(*) = "5 + E' 



L(z -ft)' 



l 



Here ft stands for nio>i -+- n 2 co 2 and 2' indicates a sum over all choices of 
ft other than = Ocox + 0w 2 ; o>i and w 2 are complex parameters, assumed 
to have nonreal ratio, and (P(z) is elliptic with periods o>i, co 2 . 
The Weierstrass Zeta Function and Sigma Function. 

(57) roo = - f poo & = - + 2' I"— !— + -i + i 

J Z a Lz — ft ft J ft 

(58) <r(z) = exp 1^(2) dz = z~[[' (l ] exp 



2-i 



z 1 /z 
Lft 2 Vft/ J 



The f-f unction is meromorphic but not elliptic; the a-i. unction is an entire 
function. 

The Jacobi Theta Functions. 

00 

e Q (z) = 1 + 22 (-l)V 2 cos 27rnz, 

w=l 

oo 

^l (z) =2^ (-l)Y n+K2)2 sin (2n + 1)ttz, 



(59) 



71=0 



2 (z) =2^ g (n+M)2 cos (2n + l)rz, 



n=0 



3 (z) = 1 + 22 ?"* cos 2rm. 

71 = 1 

Here q is a complex parameter of form e WT , where r has positive imaginary 
part. If r is chosen as o> 2 /wi, then one has the identity: 



(60) 



'{*) = 



0)1 



e'M 



exp 



T(i«i) 



Wl 



']•■© 
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There are other <r-functions a a {z), a = 1, 2, 3, defined by the equations 



(61) <r a (z) = 



?a +1 



(0) 



exp 



i(icoi) 



L CO! 



a +1 



CO! 



(« = 1, 2, 3), 



where 04 is interpreted as O - 

The Jacobian elliptic functions are expressible in terms of the ^-functions 
or in terms of the ^-functions. Let 



(62) 

(63) 
Then 



k = 



*(0)' 

L0 3 (O). 



k' = 



W) 



K = - MO)] 2 , 



h(0). 

K' = -irK. 



z 
2K 



snz = 



(64) 



Vk 



k' 



en z = 



z 
2K. 

z 
2/t 



k i z 
do 



dnz = Vk' 



2/C 



M i 



2/C 



*(z) 
03(2) 



0-1(2) 
03(2) 



0-2(2) 
0-3(2) 



Here k' = Vl — /c 2 and If, X' are related to k by eqs. (54). The functions 
sn z, en 2, dn z depend on k as required. 

For further theory of elliptic functions and integrals see Refs. 4, 5, 7, 
10, 12. 

9. FUNCTIONS DEFINED BY LINEAR DIFFERENTIAL EQUATIONS 

Attention will be restricted to the equations of second order, although 
most of the results can be generalized to equations of higher order. From 
the theory of real differential equations one knows that the homogeneous 
linear equation 



(65) 



ao(x)y" + ai(x)y' 4- a 2 (x)y = 
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has, in general, two linearly independent solutions. When the coefficients 
a,j(x) are, for example, polynomials, the solutions can be obtained as power 
series (see Chap. 5, Sect. 7). Such power series are best studied from the 
point of view of complex variables. Accordingly, one drops the restriction 
to real variables and considers a differential equation 

d 2 w dw 

(66) p (z) — + Vl (z) — + p 2 (z)w = (poOs) * 0). 

dz dz 

The coefficients pj(z) are assumed to be analytic in an open region D and 
the solutions w(z) sought are to be analytic at least in a portion of D. By 
dividing by po(z) one obtains an equation 

d 2 w dw 

(67) -7T+<Zi(z) — +Q2(z)w = 0, 

CLZf CLZ 

where the qj(z) are analytic except for poles in D. 

Series Solution at Ordinary Points. A point z of D at which all 
qj(z) are analytic is called an ordinary point of eq. (67). If the gj(z) are 
expanded in power series about an ordinary point z and w{z) is written as 
a power series with undetermined coefficients, substitution in eq. (67) 
yields a formal solution for w(z). In this solution c = w(z ) and Ci = w'(z ) 
are arbitrary, and the later coefficients are expressible in terms of c , ci by 
recursion formulas; one can write w(z) = c Wi(z) + C\W 2 {z). One can 
prove that the general formal solution converges in a neighborhood of z 
and represents an analytic solution of eq. (67). Furthermore, the general 
solution can be continued analytically throughout D, minus the poles of 
qi(z), q 2 (z), and it remains a solution under such continuation. 

Singular Points. A point z at which qi or q 2 has an isolated singu- 
larity is called a singular point of eq. (67). One can study the solutions 
near a singular point z by selecting a nearby ordinary point z\. If there 
are no singularities other than z in the circle | z — z \ < 2 1 z — Z\ \ , then 
each series solution about Zi will converge f or | z — Z\ \ < \ z — z x \ and can 
be continued analytically in the ring domain <\z — z \ < 2\z — Z\ \ , as 
far as desired; the resulting function is then multiple-valued. One can 
also seek series solutions of form 

00 

(68) (z - z y £ b n (z - z ) n , 

n = — oo 

where n is allowed to be an arbitrary complex number; such a solution 
exhibits explicitly the multiple-valued behavior near z . One can show 
that a solution of form (68) does exist. A second linearly independent 
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solution can also be found, either of form (G8) with a different choice of n, 
or of form 

00 00 

(69) (z - «o)"{log (* - «o) £ tn{z - z Q ) n + £ d n (z - z ) n }. 

— 00 ' — 00 

Regular Singular Points. If the Laurent series which appear in the 
two solutions have only a finite number of negative powers, the singular 
point z is termed regular; in this case (G8) can be written in the form 

oo 

(z - z ) a [l + £ b n (z - Zo) n ]. 

n=0 

It can be shown that z is a regular singular point precisely when 
(z — Zo)q'i(z) and (z — z ) 2 q 2 (z) are analytic at z . One can extend the 
definition to the case z = °°; this is an ordinary point or regular singular 
point according as the substitution f = 1/z leads to an equation in f having 
an ordinary point or regular singular point at f = 0. 

If every value of z, including =o, is either an ordinary point or a regular 
singular point, then the differential equation (67) is said to be of Fuchsian 
type. Since the regular singular points are isolated, there can be only a 
finite number of singular points in all, and q\{z), q 2 (z) must be rational 
functions of z. 

Hypergeometric Equation. Let eq. (67) now be an equation of 
Fuchsian type. If there are at most two singular points, it can be verified 
that the solutions are elementary functions. If there are three singular 
points, then by a linear fractional transformation z' — {az -f- b)/{cz -f- d) 
these can be placed at 0, 1, co and by a substitution w = z x (l — z) M w', the 
equation can be reduced to the form 

d 2 w dw 

(70) z(l - z) — - + {c - (a + b + l)z} abw = 0, 

dz dz 

known as the hypergeometric equation. If c is not or a negative integer, 
one solution valid for \z\ < 1 is the hypergeometric series 

a-b a(a+ 1) b(b + 1) 

(71) F{a,b,c;z) = 1+— *+ \ Q • -j—Jr* 2 + • • • ; 

1-c 1-2 c(c + 1) 

a second solution is found (for c not an integer) to be 

(72) z^Fia - c + 1, b - c + 1, 2 - c; z). 
The Legendre Equation. 

„ d 2 w dw 

(73) (1 - z 2 ) — - 2z— + n(n + l)w = 0. 

dz dz 
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This has regular singular points at ±1 and co, and is reducible to the 
hypergeometric equation. Two linearly independent solutions are found 
to be 



P n {z) =F(n+l, -n, 1; J- Jg) 

-Mn+2 2ft + 3 1 \ 



(74) ^ / x x^ 2 r(n + 1) 1 n /n+l n+2 2u + 3 1 
2 n+1 T(n + f ) z n+1 



P n (z) is known as the Legendre function of degree n of the first kind; when 
n is a positive integer or 0, P n reduces to a polynomial, the Legendre poly- 
nomial of degree n; Q n (z) is called the Legendre junction of second kind. 
The Lame Equation. 

d 2 w 1/ 1 1 

dz 2 2 \z — a z — b 



z 


1 


-) 


dw 
dz 


h + n(n 


+ 1)2 






4(z 


- a)(z - 


" &)(z - 


c) 



w = 0. 

This has regular singular points at a, b, c, oo. For details concerning its 
solutions, in particular, the Lame functions, one is referred to Chap. 23 of 
Ref. 12. 

A number of equations possessing irregular singular points are of im- 
portance in applications: 

The Bessel Equation. 

„ d 2 w dw 

(76) z 2 -j+z—+(z 2 -n 2 )w = 0. 

dz dz 

This has a regular point at 0, an irregular point at oo. One solution is the 
Bessel function of order n : 

co j j / z \n+2k 

(77) /n(z) = E(-D fc 



~ T(k + 1) T(k + n + 1) \2 

The functions J n {z) and J— n iz) provide two linearly independent solutions, 
unless n is an integer. If n is a positive integer, two linearly independent 
solutions are J n (z) and the Hankel function 

(78) r w ( g ) = i r ,, . ' 21og-+2 T - E - 

k=0 fc!(n + fc)! I 2 m =k+im) 

n ^(n - k- 1)\ /z\~ n+2k 
~ k = S W 
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where 7 = 0.5722 • • • is Euler's constant (Sect. 10). When n is not an 
integer, Y n (z) can be defined more simply: 

. J n (z) cos nx — J- n (z) 

(79) Y n (z) = 2*e B "-^ _ 

sin 2nir 

The Mathieu Equation. 

d 2 w 

(80) — - + (a + 16g cos 2z)w = 0. 

Here a and g are constants and there is one irregular singular point at 
00. The solutions are hence entire functions. For certain choices of a and 
q, eq. (80) has solutions which are periodic and either even or odd; these 
solutions are called Mathieu functions. For details on these functions one 
is referred to Chap. 19, lief . 12. 

The Confluent Hypergeometric Equation. 

d 2 w f , k 1 — m 2 } 
(8!) __ + J_ } + _ + __l w = . 

This has a regular singular point at and an irregular singular point at 00. 
One solution is given by Whittaker's function 

(82) W k , m (z) = - — r(fc + J - m)e-*'J 

2iri 

(-«)"*~ H+m fl+-j e-'dt, 

where C is a path from +» to e along the "upper edge" of the positive 
real axis, then around the circle \t\ = e in the positive direction, then from 
t = e to -f- co along the lower edge of the positive real axis. The integrand 
is single-valued if one chooses arg ( — t) to vary from — -k to +t on the 
path, if t = — z is outside C, and arg (1 + t/z) is chosen as that branch 
which — » as t — » inside C. The definition fails when fc + J^ - m is a 
negative integer, but can be modified to cover this case (see Chap. 16 of 
Ref. 12). 

10. OTHER TRANSCENDENTAL FUNCTIONS 

Other analytic functions of importance in applications are the following: 
The Gamma Function. 

(83) r(z) = J e-'t 2 - 1 dt. 
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This definition holds for Re (2) > 0, but r can be continued analytically 
and becomes a meromorphic function with poles of order 1 at 0, — 1, — 2, 
• • • . Identities satisfied by T(z) are the following: 

(84) T(z+1) =zT(z); 

(85) r(n+l)=n! (n = 1, 2, 3, • • •); 

(86) T(z)T(-z) 

(87) r(z) = lim 



z sin irz 
n\n z 



1 
(88) = ze** II 



T(z) 



z[z + 1) • • • (2 + n) 

1 + - ) e~ zln 
n/ 



where 
(89) 



/ m 1 \ 

= lim ( X) - - log m) = 0.5772 1566 49 



is the Euler-Mascheroni constant. 
The Beta Function. 



(90) 



B{z, w) = f t z ~ l {\ - W_1 dt, Re z > 0, Re w > 



This is expressible in terms of the T-function : 

T(z)T(w) 
(91) B(z, w) = -7^- 

T(z + w) 

The Incomplete Gamma Function. 



(92) 



y(a, z) = f e~H a - 1 dt, Re a > 0. 
•^0 



This is expressible in terms of the Whittaker function of the preceding 
section : 

(93) 7 («, z) = r(«) - z™ n - l) e-y**W H(n - lh y 2n (z). 
The Error Functions. 

(94) Erf (2) = f e-' 2 dt; 

Jo 

(95) Erfc (2) = I e~ l dt = - - Erf (2). 
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These functions are also expressible in terms of the Whittaker function : 

(96) Erfc (z) = &- y2 e- 1Az2 W_ HiH (z 2 ). 
The Logarithmic Integral Function. 

r z dt 

(97) li (z) = = -(- log*) > V*JP_ H ,o(- log a). 

Jq logi 

The Exponential Integral Function. 

(98) Ei(z) = -I — dt = \\{e z ). 

J —z t 

The Sine and Cosine Integral Functions. 

r z sin t 1 

(99) si z = I dt = — [Ei (iz) - Ei (-iz)L 

/* 3 sin/ 7r 

(100) Siz=j dt = - + si z; 

J Q t 2 

" cos t 



J cos t 
dt = i 
2 t 



[Ei (iz) + Ei (-iz)]. 



In eq. (97) z is first taken as real and positive, but analytic continuation 
then gives meaning to the function, as a multiple-valued function, for all 
z ^ 0. Similarly, in eq. (98), z is first to be real and negative. The func- 
tions si z and Si z are entire functions; Ci (z) — log z is also entire. 

The Riemann Zeta Function. 

(102) r(*) = Z-' Rez>l. 

This function can be continued analytically and becomes a function single- 
valued and analytic for all z except z = 1, where f(z) has a pole of first 
order. One has the integral representation: 



(103) f(s) =— f -dz, Rez > 1. 

T(z) Jo e 3 - 1 
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Remark. The purpose of this chapter is to provide a survey of opera- 
tional methods. The method which has proved most useful is that of the 
Laplace transform, which is mentioned only briefly in this chapter and is 
discussed in full in Chap. 9. 

1. HEAVISIDE OPERATORS 
Definitions. For functions of a real variable i one writes: 

d „ d 2 d n 

dt dt 2 dt n 

and defines a polynomial differential operator 

0(D) = a D n + a x D n ~ l + • • • + a„_iD + a n 
8-01 



8-02 
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by the requirement that for every fit) 
(2) 0(D)/ = (a D n +.--+a n )f = 



d n f df 

a — H h «n-l — + <W, 



wherever / has derivatives through the nth order. The coefficients o (f^O), 
Oi, • • •, a n may depend on /, but for most applications they are constants. 
The integer n is the order 'oi the operator. 

The sum and product of polynomial operators are defined by the rules: 

(3) [0i(D) + 2 (D)]/ = 0i(D)/ + 2 (D)/ 

(4) [0i(D) -0 2 (D)]/ = 0i(D)[0 2 (D)/]. 

It immediately follows that addition is the same as for ordinary poly- 
nomials; the same is true of multiplication provided the coefficients are con- 
stants. For example, 

(D + 1)(D-1) =D 2 -1, 

(D + t)(D -t) = D 2 - (t 2 + 1). 

In general, the polynomial operator is linear: 

(5) *(D)(c,/i + c 2 / 2 ) = ci*(/i) + c a *(/ 2 ), 

provided Ci and c 2 are constants. 

The reciprocal or inverse of a polynomial operator is defined by the con- 
dition : 

(6) git) = -^-Kt) if <t>(D)g(t) = f(t) 

and 0(0) = 0, g'(0) =0, • • •, £ (n_1) (0) = 0, where n is the order of 0. 
Thus [1/0(D)]/ is the solution of the differential equation 



(7) 



d n x 

«o H 

dt n 



dx 

+ a n _i — -f- a n z 

eft 



/(0 



satisfying the initial conditions x = 0, dx/d£ = 0, • • •, d n ~ 1 x/dt n ~ i = 
for £ = 0. It is assumed the differential equation is such that there is a 
unique solution satisfying the initial conditions; this is surely so if ao, ■• • •, 
a n are constants and fit) is continuous for all t. One defines the sums and 
products of reciprocal operators by the equations: 



(8) 



(9) 



L0i(D) 
1 



+ 



1 



2 (D) J 
1 



1 1 

/ = -/+-/, 

01 02 



0i(D) 2 (D) 



/ = 



1 

0^y 



i 



.02(D) J 
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Addition is commutative: (l/^i) + (l/fo) = (1/02) + (l/«£i), as is mul- 
tiplication, provided the coefficients are constant. 

The ratio of two polynomial operators is defined by the equation: 



(10) 



4>i{D) 
4>2(D) 



f = MD) 



MD) f l 



Here the order chosen is essential: it is not true that 



(11) 



<t>i(D) 



U 2 (£>) 



/ 



4»(D) 



[4>i(D)f], 



even if the coefficients are constant. All operators defined thus far are 
linear. 

Integral Representation of Inverse Operators with Constant 
Coefficients. One has the formulas: 



(12) 
(13) 
(14) 



D 



1 r l 

-fit) = f(u) du, 
D Jo 

i , r l 

f{t) = e at e~ au fiu) du, 

-a J 

1 e~ au (t - uf- 1 



(D - a)' 



at C 

Jo 



f(u) du, 



where a is constant and k = 1, 2, 
algebra: 



(fc-1)! 
Now (j)(D) can be factored as in 



(15) 4>(D) = a (D - r,)(D - r 2 ) • • • (D - r„) 
where r\, • • •, r n are the roots of the characteristic equation 

(16) 0(r) = a r n + • • • + a n ^r + a n = 0. 
Correspondingly, 



(17) 

Thus if n = 2 

1 

■f = 



1 



*(£) 



/ = 



1 



a (D - n) • • • (D - r„) 



/• 



1 



•/ 






0(D) ' a (D - n) L(JD - r 2 ) 
In general, computation of [1/<£(D)]/ is reduced to a repeated integration. 
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If (j>(r) has complex roots, quadratic factors appear in eq. (17) ; for these one 
has the rule : 

„at n t 



1 

(D - a)* + b' 2 



(18) - -f = — I e~ au sin b{t - u)f(u) du. 

oJq 



If 1/4>{D) is expanded in partial fractions as in algebra, then the corre- 
sponding operator identity is valid; for example, 

D 2 -l \2D-1 2 2) +1/ 

= |e< I e~ u f(u) du - \e~ l I e u f(u) du. 
Jo Jo 

Heaviside Expansion Theorem. More generally, a ratio fa(D)/fa(D) 
can be replaced by its partial fraction expansion. If in particular the de- 
gree of fa exceeds that of fa, and fair) has simple roots r\, r 2 , • • •, r n , 
then by Chap. 7, Sect. 5, 

fa(r) A«ifa) 1 



fair) fc=i fain) r - r k 
(19) 

fa(D) = » fa(r k ) 1 



02(2)) k=i<f>'2(r k ) D - r k 

This is in essence the Heaviside expansion theorem. 
Power Series Operators. The formal relation 

1 1 1 D D 2 



D - a { D\ a a 2 a 3 

does not agree with the definition of 1/(2) — a). However, if the operator 
is applied to a polynomial in t, one obtains a particular solution of the 
corresponding differential equation (with modified initial conditions). For 
example, 

/ 1 D D 2 \ 2 _ t 2 2t 2 

\ a a 2 a 3 / a a 2 a 3 

is a solution of 

dx 

ax = t 

dt 

for which x(0) = — 2a~~ 3 . 



OPERATIONAL MATHEMATICS 8-05 

One can also expand in inverse powers of D : 



(2°) " = - + ^ + ^~ 



1 1 

In this case the rule can be proved to be correct. 



The power series ^Z h n D n /n\ can be interpreted as the operator e hD . 
o 
One then finds, under appropriate conditions, 



(21) e hD f = £ — ^ hn = W + *)■ 

o n\ 



2. APPLICATION TO DIFFERENTIAL EQUATIONS 

The general solution of a linear differential equation, 

(22) 4>(D)x = f(t), cj>(D) = a D n + • • • + a n , 

is formed of the complementary function x c (t), which is the general solution 
of the homogeneous equation 4>(D)x = and of a particular solution x p (t) 
of the given equation : 

(23) x = x e (t) + x p (t) 

[cf. Chap. 5, Sect. 3]. The Heaviside operators provide simple ways of 
finding x p (t), namely as the function 

(24) x p (t)= ——fit); 

this is the solution with zero initial conditions. If 1/4>(D) is expanded in 
partial fractions, one can then apply the integral formulas (12), (13), (14), 

(18). 
The procedure can be extended to simultaneous equations. For example, 

Dx+{D - \)y = F(t) 

(D + l)x + 2Dy = G(t) 

can be solved formally: 

2D D - 1 D D + 1 

x = — F(t) - — G(t), y = — G(t) - — F(t) 

D 2 +l D 2 + l D 2 + l D 2 + l w 

and it can be verified that these provide the solution for which x = 0, 
y = when t = 0. 
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3. SUPERPOSITION PRINCIPLE. RESPONSE TO UNIT FUNCTION AND 
DELTA FUNCTION 

The Heaviside unit function u(t) is defined to equal for t < and to 
equal 1 for t ^ 0. The solution of the differential equation 4>{D)x = u(t) 
with zero initial values, i.e., the function (l/cf>(D))u(i) = A(t) is known as 
the indicial admittance or step response. 

The superposition principle states that the response of a linear system to 
a linear combination C\fi(t) + c 2 f 2 (t) equals the corresponding linear com- 
bination CiXi(t) + c 2 x 2 (t) of the responses Xi(t) to fi(t), x 2 (t) to f 2 (t). In 
the typical case x(f) and/(0 are related by a differential equation <f>(D)x = 
f(t) and the superposition principle is equivalent to the statement that 
l/<f>(D) is a linear operator. 

One can apply the superposition principle to show that (when <f>(D) has 
constant coefficients) the response to a general f(t) is deducible from the 
indicial admittance, i.e., the response to u(t). Indeed, the response to 
u(t — h), for h ^ Q,is A(t — h); one can approximate f(t) by a linear com- 
bination 2kCku(t — tk), where c& = f(tk+i) — f(tk). A passage to the limit 
gives the Duhamel theorem 



(25) x(t) 



= f f(s)A'(t-s)ds. 
Jo 



[It is assumed that f(t) is for t < and the solution x(t) has initial 
values]. If f(t) is constant, equal to 1/e for ^ t ^ e, and then equal to 
for t > e, the response is [A(t) — A(t — e)]/e. The limiting case of such 
an/(i), as e — > 0, is an "ideal function," the delta function 8(t), also termed 
the unit impulse function. The response to 8(t) is interpreted as A'(t) = 
h(t). Accordingly, 

(26) x(t) = f f(8)h(t - s)ds. 

Jo 

For some linear systems the response to u(t) appears as [<f>(D)/^(D)]u, 
where <f> and ^ are polynomials. If \p has simple roots b a (a = 1, • • •, k), 
then by eqs. (19) 

0(D) * *(&„) 1 * *(6„) e b ^ - 1 

u = y. u = y, u(t) 

HD) a ±[V(b a )D-b a „tl^(6«) b a 
and hence 

^ ,A ^ ,A , £ *W eM /A 

(27) — - it(0 = — u(t) + £ — — — u{t). 

f(D) ^(0) o=llA(6«) &a 
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4. APPRAISAL OF THE HEAVISIDE CALCULUS 

The operational methods described in the preceding section provide a 
valuable tool for solution of linear differential equations. The method has 
two principal drawbacks : it is very awkward to obtain solutions with speci- 
fied initial values other than 0; further development of the method leads 
to symbolic expressions whose meaning has to be studied afresh in each 
case. Great ingenuity has been employed to remedy these defects but a 
satisfactory general theory within the Heaviside framework has not been 
found. 

On the other hand, it has been discovered that all the goals of the 
Heaviside calculus can be achieved without reference to differential opera- 
tors or their inverses and, indeed, without any symbolic calculus. The 
means to this end is the Laplace transform (see Chap. 9) ; the closely related 
Fourier transforms can also serve the purpose. By means of these the 
questions about initial conditions are easily disposed of, and justification 
of formal rules becomes simple. 

The transformations referred to do not merely serve as a substitute for 
the Heaviside calculus. Deeper study shows that they lie at the very basis 
of that calculus and must inevitably enter in a full justification of the 
operational rules. 

5. OPERATIONAL CALCULUS BASED ON INTEGRAL TRANSFORMS 

One considers equations of form 

(28) F(y) = f f(t)K(t, y) dt. 

J a 

Such an equation assigns a function F{y) to each function f{t), whenever 
the integral has meaning. One calls F the integral transform of / with re- 
spect to the particular transformation (28) and writes: 

(29) F = T[f]. 

The relation between / and F is much like that between independent and 
dependent variables; here the variables are functions. 

Because of the form of eq. (28), the transformation T is linear: 

(30) T[c x f x + c 2 / 2 ] = Cl T[h] + c 2 T[f 2 ]. 

The transformation (28) is said to have a (single-valued) inverse if, for 
each F of a certain class, there is precisely one / such that T[f] = F. One 
writes : 

(31) / = T~\F\ 
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and calls / the inverse transform of F. Because T is linear, T~ l must also 
be linear. 

Convolution. If to each pair of functions /i, f 2 one can associate (in a 
unique manner) a third function / 3 such that 

(32) T[f 3 ]=T[f 1 ]-T[f 2 ], 
then one calls / 3 the convolution of f\, f 2 and writes: 

(33) /s=/i*/ 2 . 
The convolution must then obey simple laws: 

, N /i*/2=/2*/i;/i*(/2+/ 3 ) =/i*/ 2 +/i*/ 3 ; 

(34) 

/i * cf 2 = cf x */ 2 = c(/i */ 2 );/i * (/ 2 */ 3 ) = (/i *h) * h- 

Solution of Differential Equations. Suppose the transformation 
T has the property that, for a certain polynomial differential operator 0(D) 
and for f(t) in a certain class of functions, one has an identity : 

(35) T[4>(D)f] = H(y)T[f] = H(y)F(y), 

where H{y) is a function of y associated with the operator 0. Then to 
solve a differential equation 

(36) 4>(D)x = g(jt) 

for x = f(t) in the class referred to, one forms the transformed equation 

T[<t>(D)x] = T[g] 
or equivalently, by eqs. (35), 

(37) H(y)F(y) = G(y). 

Accordingly, 

G(y) , [G(yY 

H{y) lH(y). 

One can try to find the inverse transform of G(y)/H(y) with the aid of 
tables of functions and their transforms. One can also seek 



(39) T 



— i 



1 



Lff(y)J 



= w(t). 



Then eq. (38) gives 

(40) T[f(t)] = T[w(t)] ■ T[g(t)] = T[w * g], 
so that 

(41) f(t) = w(t) * g(t). 
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The crucial question is choice of the transformation T so that eqs. (35) 
hold. For differential equations with constant coefficients it is sufficient 
to choose T so that 

(42) T[Df(t)] = H(y)F(y). 
For then 

(43) T(a D n + • • • + an^D 4- a n )f = (a H n + • • • + a x H + a n )F(y). 

Fourier Integral. Now associated with the operator D are certain 
functions / such that Df is a constant times /; these are precisely the func- 
tions ke at . It is known that an "arbitrary" function / is expressible as a 
"sum" of functions of this form. For example, under appropriate condi- 
tions, 

(44) f(t) = f F^e^da; 

J —CQ 

this is the representation of / as a Fourier integral. One finds that 

(45) F(u) =— I Me-*«dt, 

2ir J_oo 

so that F(o)) can be considered as T[f], a linear integral transform of T; 
except for a constant multiplier, this is the Fourier transform of /. The 
fact that De lut = ie l0it is reflected in the formula •' 

(46) Df = f'(t) = f iwFMe™* do>, 

which follows from eq. (44). Hence 

(47) T[Df] = i<*F(fS). 

Thus the transformation T defined by eq. (45) has the property desired. 

The functions / representable as Fourier integrals must be small for 
large positive or negative t (see Sect. .9). For functions not satisfying such 
a condition other representations can be used. If / is defined only for 
t ^ and does not grow too rapidly as t — -> w, one can use the Laplace 
transform. If / is defined for all t and has period 2ir, then / can be repre- 
sented by a Fourier series ; associated with this series is the finite Fourier 
transform. 

If <f>(D) does not have constant coefficients, the transformation T must 
be related specially to the particular operator <£. Associated with tf> are the 
"characteristic functions" / for which <j>D(f) is a constant times/. Repre- 
sentation of an arbitrary function as a series or integral of such character- 
istic functions leads to a corresponding integral transformation. 
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6. FOURIER SERIES. FINITE FOURIER TRANSFORM 

Let f(t) be defined for all real t. One says that fit) has period T ^ if 
Jit + T) = fit) for all t. A function fit) given only for a < t < b can 
always be defined outside this interval so as to have period T = b — a 
(periodic extension of f(t)). 

Let f(t) have period T and let w = 2-k/T. The Fourier series of fit) is 
defined as the series: 



(48) 

where 

(49) 



a 



+ S ( a n cos ncot + b n sin wcof), 



2 r T 2 r T 

a n = — I /(0 cos ncof d£, b n = — I /(f) sin na>£ rff. 

T J ^ *^o 



Because of the periodicity of /(/), the interval of integration in eqs. (40) 
can be replaced by any other interval of length T, e.g., from — Y^T to %T. 





*' 


k 










/ • 


y i 


1 / • 


I 


• 


/ •/ 

l 










T 




2T 


t 



Fig. 1. Piecewise continuous function of period T. 

It is assumed that the integrals in eqs. (49) have meaning. For this it is 
sufficient that fit) be piecewise continuous, i.e., continuous except for jump 
discontinuities (Fig. 1). 

Convergence. The Fourier series of f(t) converges to fit) under very 
general conditions: for example, wherever f(t) is continuous and has a 
derivative to the left and to the right. At a jump discontinuity t the 
series converges to 

where 



(50) 



/(<o+) = lim f(t), 



fit -) = lim f(t), 
t-*t n - 
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provided [f(t + h) — f(t +)]/h and [f(t — ) — /(/ — h)]/h have limits as 
h — > 0+. For example, if fit) = t for — 1 < t < 1 and /(£) has period 2, 
then the corresponding Fourier series converges to at t = I, t = — 1, 
£ = 3, £ = —3, • • •. It is common practice to redefine fit) as 3^[/(^o+) + 
/(^ — )] at each jump discontinuity. 

If f(t) is merely continuous, there is no general theorem on convergence. 
However, one has a "convergence in the mean," that is, if s n (t) denotes 
the sum of the first n terms of the series (48), then the "mean square error" 

~ f U(t) - s n (t)] 2 dt 

1 Jo 

tends to as n — > oo. This result holds considerably more generally, 
e.g., if / is merely piecewise continuous. 

If/(0 nas a continuous derivative over an interval t ^ t ^ t\, then the 
Fourier series of fit) converges uniformly to fit) over this interval; i.e., 

(51) max |/(0 — s n (t) | — > as n — > oo. 

In general, if a series of form (48), i.e., a "trigonometric series," converges 
uniformly to fit) f or ^ t ^ T, then the series must be the Fourier series 

of/(f). 

A function is determined uniquely by its "Fourier coefficients" a , ai, 
• • •, bi, • • • ; that is, ii fit) and g(t) have the same Fourier coefficients, then 
f(t) = g(t) except perhaps at points of discontinuity. 

Fourier Cosine and Sine Series. If f(t) is even [fit) = f( — t)], then 
all b n are and f(t) is represented by a Fourier cosine series; that is, 

«o " 4 r T > 2 

(52) fit) = h Z~i a n cos nut, a n = — I f(t) cos nut dt, 

2 n =i T Jo 

provided the convergence conditions are satisfied. If f(t) is given merely 
between and y&T, eqs. (52) are still valid; for/(0 can be extended to all t 
to be even and have period T. Similar remarks apply to representation of 
an odd function [fit) = —f( — t)] by a Fourier sine series: 

(53) f(t) = ]C °n sm no >t, b n = — I f(t) cos nut dt. 

n=l T *^o 

The identities: 

, . 1 . 

(51) cos a = J(e ta + e~ ta ), sin a = - (e ia - e -* a ), 

2i 
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lead to a rewriting of the formulas (49) in complex form. Under conditions 
for convergence, 

1 r T 
(55) f(t) = £' c n e in ^, c n = - I f^e-™" 1 dt, n = 0, ±1, • . •. 

n=— oo * •'O 






Finite Fourier Transform. One can interpret the doubly infinite 
sequence of numbers 

,t 
f(t)e- in " 1 dt, n - 0, ±1, ±2, ••-, 

o 

as a function of n, <$>(n), defined only when n is an integer. The equation 

T 

(56) 0(n) = f /(/)e- lW * 

can then be regarded as a special case of the linear integral transformation 
(28); the variable y is replaced by n and is restricted to integer values. 
The notations: 

(57) 0(n) =*[/(«)] or = *[/] 

will be used to denote the functional transformation <fr, the finite Fourier 
transformation, which assigns the function <£(n) to the function /(f). $ is 
then defined at least for all f(i) which are piecewise continuous for ^ t 
^ T. 
As in Sect. 5, <& is linear: 

(58) $[ Cl /i + c 2 / 2 ] = Cl *[/i] + c 2 3>[/ 2 ]. 

Inverse Transform. If $[/] = 0, then one writes: / = <l>~ 1 [0]. The 
inverse transformation is then uniquely defined by the theorem stated 
above concerning functions having the same Fourier coefficients. It is a 
less simple matter to describe those functions <j>(ri) for which <& — 1 exists. 
One class of such functions 4>{n) consists of those for which the series 
'2T~ 1 (j>(n)e inut converges uniformly f or ^ t ^ T. The sum of the series 
is then a function /(0 which serves as $ -1 [^]; 

(59) *~ 1 [0] = - £ *(n)e' B " f . 

■* n = — oo 

Convolution. Given fi(t), / 2 (f) having period T, their convolution is 
defined as: 

T 

(60) hit) = f his)f 2 it - s) ds; 

Jo 
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one writes: f 3 (t) = fi(t) * f%(t). One can then prove the characteristic 
property : 

(61) *[/i*/ 2 ] = $[/i] • *[/ 2 ]. 

Transformation of Derivatives. If f(t) has a continuous derivative 
for ^ t ^ T, then an integration by parts proves that 

(62) *[/'(<)] = f(T) - /(0) + ino&[f(t)]. 

This rule can be made the basis for application of the finite Fourier trans- 
form to boundary value problems. Interest will be concentrated here on 
the periodic case:/(7 1 ) = /(0), for which the rule becomes 

(63) *[/'(«)] = inc&\f(t)]. 

Similarly, if / is periodic and has continuous derivatives through the fcth 
order, 

(64) *[/ (t) (01 = (in^[f(t)]; 

this relation remains true if / (fc-1) (0 is continuous and / (fc) is continuous 
except at a finite number of points at which left and right hand kih deriva- 
tives exist. From eq. (64) it follows that, for every polynomial operator 
ip(D) = aoD n + • • • + a n _iD + a n with constant coefficients 

(65) *{iKD)[/(0]| =*(«*))*[/(«)]. 

Steady-State Solutions of Differential Equations. Let/(0 be piece- 
wise continuous and have period T. Let a , • • • , a n be constants and let 
\f/(D) = a D n + • • • + a n —iD + a n . It can then be shown that in general 
the differential equation 

(66) HD)x = f(t) 

has a solution x = X(t) having period T; X(t) has continuous derivatives 
through the (n — l)st order and an nth derivative which is continuous 
where F(t) is continuous. If \p{y) has no root of the form inu for some n, 
there is precisely one such periodic solution; it will be assumed in the fol- 
lowing that \f/(ino)) ^ for every n. Applying the finite Fourier trans- 
formation to eq. (66), one finds by eq. (65) 

(67) 5>[X] = -^r—> where </>(n) = *[/]. 

■>}/{inu>) 
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This equation gives the Fourier coefficients of X(t), from which one can 
write 



(68) 



X 



1_ " 4>(n) 



One can attempt to reduce this to a simpler form by developing a table of 
finite Fourier transforms and inverse transforms. One can also apply the 
convolution formula to eq. (67) : 



(69) 



X = g*f= f f(s)g(t - s) ds, 



where g = $ — 1 [l/\p(inco)]. To find g, decompose l/\f/(inoo) into partial frac- 
tions and apply linearity. The problem is reduced to finding inverses of 
(ina) — a)~ k (k = 1,2, • • •)• One finds: 



(70) 



$ 



$" 



$" 



— i 



1 



into — a. 

1 

{inu — a) 2 . 

1 



= k a e at , 

= e at [k a t + Tk a 2 e dT ], 
1 d r ' 



_(ina) — a) 



— a\ m + 1 



m ! da r ' 



(k a e at ), 



where k a = (1 — e aT ) l . In particular, if \f/(p) has simple roots p 1} 
p m , so that 



(71) 

one finds that 

(72) 

where 

(73) 



1 



= z 



Hp) j=i v - pj 



X(t) = T l A i H f (t t p i ) 

3 = 1 

H f (t, v) = e pt [Q f (t, p) + k p e pT Q f (T, p)], 



QA 



,P) = fm 



e~ ps ds, ^ t^ T. 



The operators Q/ and Hf can be tabulated for various functions f(t) of 
interest, so that the corresponding periodic solutions can be found easily. 
For tables and illustrations of applications see Ref. 11. 
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7. FOURIER INTEGRAL. FOURIER TRANSFORMS 

Fourier Integral. By allowing the period T to become infinite, one is 
led to the following integral analogue of the Fourier series expansion: 

fit) = I [a(co) cos cot + b(u>) sin cot] dco, 
Jo 
(74) 

1 r 00 1 r™ 

a(co) = - I j(t) cos oit dt, b(o)) = - I j(t) sin cot dt. 

7T J_ o0 7T J_ 00 

The "coefficients" a(«), 6(w) exist if /(0 is, for example, piecewise continu- 
ous, and I | j(t) | dt exists. The representation of j(t) as a Fourier inte- 

J — 00 

gral is then valid under the same conditions as for Fourier series, e.g., 
wherever /'(0 exists. Also, under the conditions described in Sect. 8, the 
integral equals HLf(^o+) + /(fo — )] at each jump discontinuity of/. One 
can write eqs. (74) in complex form: 

(75) f(t) = f A^e*" do, A(«) = — f Me-^'dt; 

"'-00 27rJ_ 00 

the first integral must, however, be treated as a principal value, i.e., as 

lim I • • • as b — » <». 

Under conditions analogous to those for Fourier series one is led to repre- 
sentation of a function j(t) in the interval ^ t < <x> by a Fourier- cosine 

Jt 00 
a(u>) cos cot dco. It is customary to define the Fourier cosine 
o 

transform of j(t) as 

(76) F e (u) = - j f{t) cos at dt 

so that the Fourier cosine integral representation of / reads 

(77) fit) = I- J F c (w) cos cot dco; 

thus / is also the Fourier cosine transform of F c . Similar formulas hold 
for the Fourier sine transform: 



(78) FM = /- 



h I fit) sin "t dt, jit) = /-I F s (co) sin cot dco. 
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Similarly one defines the (exponential) Fourier transform of/ as 

(79) F( a ) = ^f ' Me-^'dt, 
so that by eqs. (75) 

(80) m = -2= f f ( a)e i»t fa 

Properties of the Fourier Transform. For simplicity, the numerical 
factor is dropped and the Fourier transform is defined as 

(81) <M/] = f f(t)e-^dt; 

then $,»[/] is a linear operator. If / has a continuous derivative f'(t) and 
/(Oj /'(0 satisfy the conditions stated above, then 

(82) *J/'] = i^Ml 
A convolution is defined as follows: 

(83) f*g= f f(s)g(t - s) ds = h(t) 
and one has the characteristic property : 

(84) *Jf*g] =*»UY*M; 

it is assumed here that /, g satisfy the conditions given above. An inverse 
operator is defined by the condition: <b x ~ 1 [F] = /, if ^> x [f] — F. The func- 
tion / can be shown to be uniquely defined by its transform F. 

The applications of the Fourier transform to differential equations paral- 
lel those for the finite Fourier transform, as described in Sect. 8 above; eq. 
(65) is replaced by 

(85) *«,[iKD)/(0] = *(*«)*«[/]. 

Application of the transform to the equation \}/{D)x = f(t) yields a solution 
in the form of a Fourier integral : 

(86) X(t) =- -^ e™< do:, F(u) = *.[/], 
or as a convolution: 

(87) X(t)=f*g, g = $ x 



Miud . 
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If /(0 = for t < 0, the same solution is obtainable by Laplace transforms; 
see Sect. 10 and Chap. 9 below. 

References to tables of Fourier transforms are given at the end of this 
chapter (Refs. 1, 5, 6). 

8. LAPLACE TRANSFORMS 

The Laplace transform of fit), t ^ 0, is defined as 

(88) F(s) = L[f] = f f(t)e~ st dt. 



It is convenient to allow s to be complex: s = a + iu. Equation (88) then 
reads: 

(89) F(cr + ia>) = L[f] = f f{t)e~ a ^~ iat dt; 



hence for each fixed a the Laplace transform of / is the same as the Fourier 
transform of f(t)e~ at , where / is considered to be for t < 0: 

(90) W] = *»U(t)e-«\. 

Accordingly, the Laplace transform is well defined if a is chosen so that 

(91) f |/(0 I e- 1 d« 

exists, and for such a one can invert: 

1 r™ 

f(t)e- at = — I F(a + iu)e tut da, 

2tT t/_oo 



(92) 



i r™ 

f{t) = L~ 1 [F(s)] = — F(s)e st du, t > 0; 



in the last integral s = a + ica, a has any value such that (91) exists, 
and the integral itself is a principal value. The integral can be interpreted 
as an integral in the complex s-plane along the line a = const., w going 
from — » to +<x> (Fig. 2). Since ds = idu on the path, 

(93) f(t) = — fFis)e st ds, 

2tti Jc 

C being the line <x = const. The conditions for equality of left and right 
sides of (93) are the same as for Fourier integrals. At t = 0, fit) will 



8-18 



GENERAL MATHEMATICS 



in general have a jump, because of the convention that f(t) be f or t < 0, 
and accordingly the right hand side gives 3 / £f(0+). 

The validity of eqs. (88) and (92) depends on choosing a so that (91) 
exists. It can be shown that for each/(£) there is a value o- , — °° ^ o- ^ 



A CO 
s-plane 



°o 



Fig. 2. Path of integration for inverse of Laplace transform. 

+co, called the abscissa of absolute convergence, such that the integral (91) 
exists for a- > o- . If <r = — «, all values of a are allowable; if a = + <», 
no values are allowed. 

Further properties of the Laplace transform and its applications are dis- 
cussed in Chap. 9. 

9. OTHER TRANSFORMS 

The two-sided Laplace transform is defined as 



(94) 



LAf] = F(s) = f f(t)i 

J — m 



dt. 



Hence it differs from the (one-sided) Laplace transform only in the lower 
limit of integration; thus 



(95) 



Lilf] = *,o[/(0e-"], 



with no requirement that f(t) be for t < 0. The two-sided transform is 
thus a generalization of the one-sided transform. 
The Laplace-Stieltjes transform of g(t) is defined as 



(96) 



G(s) 



-f 



dg(t). 



The integral on the right is an improper Stieltjes integral; it has meaning 
if g(t) is expressible as the difference of two monotone functions and if the 
limit as b —* +00 of the integral from to b exists. If g'{t) = f(t) exists, 
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then G(s) is the Laplace transform of f(t). If g(t) is a step function with 
jumps at t-i, t 2 , • • •, the integral reduces to a series 2cje~~ tiS . For further 
information one is referred to the book of Widder (Ref. 10). 

Other integral transforms have been defined and studied. These have 
found their main applications in the boundary value problems associated 
with partial differential equations; they could conceivably be applied to 
ordinary linear differential equations with variable coefficients, on the basis 
of the analysis of Sect. 5. 

The Legendre transform is an example which assigns to each f(t), — 1 ^ 
t ^ 1, the function 



(97) 



T[f] = *(n) =j f(t)P n (t) dt, n = 0, 1, 2, 



where P n (t) is the nth Legendre polynomial. The transformation has the 
property 

T[R{f}] = -n(n+l)0(n), 



(98) , , d 

R{f} = 



(1 " t 2 ) 4/(0 



dt I dt' 

Hence the transform can be applied to differential equations of form 

(99) (a R m + a x R m ~ l + • • • + a m _^ + a m )x = /(*), -1 ^ t ^ 1, 

where Oo, ••■,«„, are constants. For details on the Legendre transform 
see Ref. 12. 

The Mellin transform, Bessel transforms, Hilbert transform, and others 
are defined and their properties are listed in the volumes of the Bateman 
project (Ref. 1). 
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1. FUNDAMENTAL PROPERTIES 

Of the various operational methods described in Chap. 8 those based on 
the Laplace transform have proved to be the most fruitful. 

Basic Definitions and Properties. Let f(t) be a function of the real 
variable t, defined for t ^ 0. The Laplace transform of f(t) is a function 
F(s) of the complex variable s = a + iia: 

(1) Uf] = F(s) = f f(t)e~ st dt. 

9-01 
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It is convenient to allow f(t) itself to have complex values: f(t) = fi(t) + 
#2(0 , though for most applications/ will be real. 

It will be assumed that f(t) is piecewise continuous (Chap. 8), although 
the theory can be extended to more general cases. It can be shown that 
there is a number a , — 00 ^ o- 5S +00, such that 



(2) 






1/(0 



dt 



exists for <r > <ro and does not exist for a < cq. If o- = — °°, the integral 
exists for all <r; if o- = +°o, it exists for no <r; o- is called the abscissa of 
absolute convergence of L[f]. If a > <r , then the Laplace transform of / 
does exist. Accordingly, there is a certain half-plane in the complex s-plane 
for which L[f] = F(s) is defined (Fig. 1). Furthermore, F(s) is an analytic 
function of s in this half-plane (Chap. 7, Sect. 2). 

Remark. For existence of F(s), it is sufficient that the integral in (1) 




Fig. 1. Domain of definition of F(s) = L[f]. 



have meaning. It can be shown that there is a number <r\, the abscissa of 
(conditional) convergence, for which this integral exists, and a\ ^ o- . For 
most applications v\ = a and for most operations on F(s) it is simpler to 
restrict a to be greater than o- . 

Examples or Laplace Transforms. These are given in Table 1. For 
extensive tables one is referred to Refs. 1, 5, 6, Chap. 8. 

Existence. For practical purposes the condition that the Laplace 
transform exist for some a is that the function f(t) should not grow too 
rapidly as t — *■ +00. For example, e , e e , do not have Laplace transforms. 
In general, a function of exponential type, i.e., a function for which \f(t)\ 
< e kt for some k and for t sufficiently large, has a Laplace transform 
F(s). 
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Linearity. The Laplace transform is a linear operator. More precisely, 
if L[/i(0] = Fi(s) exists for a > <x\ and L[f 2 (t)] = F 2 (s) exists for a > a 2 , 
then for every pair of constants c x , c 2 L[c\f\ + c 2 f 2 ] exists for a > 
max (<ri, <r 2 ) and 

(3) L[ Cl h + c 2 / 2 ] = ciLL7i] + C2L[/ 2 ]. 

2. TRANSFORMS OF DERIVATIVES AND INTEGRALS 
Rules. 

(4) L[f'(t)] = sL[f(t)]-f(0), 

(5) « = s*L[f(t)] - f (0) - s/(0), 

(6) L[/ (n) (01 = s n L[/] - [f n - l) (0) + S / (n " 2) (0) + • • • + s n ~7(0)], 



(7) L [ J" /(0 dt 



= -L[f]. 
s 



The first rule is basic here, the others being consequences of it; it is valid 
if (for some a) f(t) and f'(t) have Laplace transforms and f(t), f'(t) are 
continuous for t ^ 0. More generally, eq. (4) is valid if only f(t) is con- 
tinuous and f'(t) is continuous except for jump discontinuities. Similarly, 
eq. (6) is valid if the Laplace transforms exist and all derivatives concerned 
are continuous except perhaps the nth, which is allowed to have jump dis- 
discontinuities. Rule (7) is valid if / is piecewise continuous and the trans- 
forms exist. 

Example. If / = sin t, f = cos t, /(0) = 0, so that L[cos t] = sL[sin t] 
= s/(s 2 + 1). 

Of great importance is the special case of eq. (6) : 

(8) L[f n \t)] = s n L[f], if /(0) = /'(0) = • • • = /'—"(O) = 0. 

Hence, if one restricts to functions with initial values, differentiation with 
respect to t corresponds to multiplication by s. 



'(0 



1 


1 


2 


e at 


3 


t n {n > -1) 


4 


t n <? 1 (« > - 1) 


5 


cos a£ 


6 


sin a£ 


7 


cosh a? 


8 


sinh at 


9 


t n cos at(n> — 1) 


10 


f n sin at (n > — 1) 


11 


cos 2 £ 


12 


sin 2 £ 



13 sin at sin &£ 



Table 1. Laplace Transforms 

F(s) = L{f] = ff(t)e-°< dt 

Jo 



O 

4^ 



1/S 

l/(s - a) 
r(n + 1) 



or, if n = 0, 1, 2, 



T(n +1) w! 

(7^^,1^ = 0,1,2,..., (s _ a)w+i 



s/(s 2 + a 2 ) 








a/(s 2 + a 2 ) 








s/(s 2 - a 2 ) 








a/is 1 - o z ) 








r(n + 1) (s + ai) n 


+1 + (s - 


ai) r 


+i 


2 ( 


s 2 + a 2 )"+ 


i 




r(n + 1) (s + oi) n 


+1 - (s - 


ai) r 


i+i 


2i (s 


J + fl 2)»+l 






-(' + S ) 
2 \s s 2 + 4/ 








1/1 s X 








2 Vs s 2 + 4/ 




2a6s 









[s 2 + (a + 6) 2 ][s 2 + (a - 6) 2 ] 



Range of cr 
cr > 
a > Re (a) 

cr > 

a > Re (a) 

cr > | Ira a | 

o- > |Im a | 

a > \ Re a | 

cr > | Re a | 

a - > | Im a | 
cr > | Im a | 

cr > 
CT> 

o- > Max (a, 0) 
a = | Im (a + 6) | 
|8 = | Im (a - 6) | 



O 



7J 
> 



> 

x 



n 



14 e at sin (6/ + c) 



1. * ^ 1 



2 - *, 1 ^ t£ 2 



(s — o) sin c + & cos c o- > Max (a, /3) 

(s - a) 2 + 6 2 <x = Re (a + bi) 

jS = Re (a - &i) 



15 1 for 2n ^ < < 2n + 1 _ a > 
for 2n + 1 ^ « < 2n + 2 s(l + e ) 

n = 0, 1, 2, • • • (square wave) 

16 1 for a ^ t ^ 6 < °o : all <r 

for ^ * < a and * > b 

17 0for0^«'<&, - — o- > 

1 for f > b 



s 




e -os 
S 




l-e~ 


^ 


s 2 




(1-e- 


- s) 



18 t,Q£tgl = — a > > 



n 



19 t,0£t£l 2—^- all o- 3 



> 

0, ^ 2 S 

20 a-£|«-(2n + l)&| r^f^ cr > 1 

OS j/j 

for 2n6 ^ f ^ (2n + 2)6, 
w = 0,1, ■■-,&> 0, 

a real (triangular wave) 

a (\ -4- 6s e 6s ) 

21 o(< - n&) for nb ^ t < (n + 1)6 _ o- > 

(sawtooth wave) s ^ e ' 



o 
en 



9-06 



GENERAL MATHEMATICS 



3. TRANSLATION. TRANSFORM OF UNIT FUNCTION, STEP FUNCTIONS, 
IMPULSE FUNCTION (DELTA FUNCTION) 

Translation. In Laplace transform theory it is convenient to consider 
each function f(t) to be defined as f or t < 0. Hence for c ^ f(t — c) is 
for t < c and coincides with a translated /(f) for t > c (Fig. 2). One finds 



(9) 



L[f(t ~ c)] = f f{t - c)e~ st dt 

J c 

= e~ cs L[f]. 



XI 


> 








f(t) 


fit - c) 






< 




t 



Fig. 2. Translated function. 

Unit Function. Now let u(t) = for t ^ 0, u(t) = 1 for t > 0; w(0 is 
called the unit function (of Heaviside). By entry 1 of Table 1, 



(10) 
Hence for c 

(ID 



L[u(t)] = -• 
s 



L[u(t - c)] = 



where u(t — c) is the translated unit function with jump at t = c (Fig. 3) ; 
cf. entry 17 of Table 1. A square pulse of height h (Fig. 4) can be repre- 



xk 



u(t - c) 



xk 



Fig. 3. Translated unit function. 



Fig. 4. Square pulse. 
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sented as a combination of two unit functions: 

(12) f(t) = h[u(t - a) - u(t - b)], ^ a < b; 

hence its transform is 

(13) 
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h 



L[f] = - (e~ as - e~ ba ). 
s 



X' 


, 










































I 
i 

i 














t 






a 


l 


a 


2 


a 


3 


t 



Fig. 5. Step function. 

A general step function (Fig. 5) can be regarded as a superposition of such 
square pulses: 

(14) / = hi[u(t) — u(t — ai)] + h 2 [u(t - a{) — u(t — a 2 )] -\ ; 

hence (if the pulses do not grow too rapidly, so that L[f] exists) 

(15) L[f] = - [h(l - e" a > 8 ) + Me - " 18 - e" a ' s ) +•••]• 

s 

Impulse Function. The unit impulse function at t = is defined as 
the limit as e — » of a square pulse form t = to t = e and having unit 
area, i.e., the limit as e — > 0+ of 



(16) 



1 



[u(t) - u(t - «)]. 



The limit does not exist in the ordinary sense; it can be considered as de- 
fining an "ideal" function, the delta function 8(t). One can consider 8(t) 
to be except near t = where 8(t) is large and positive and has an integral 
equal to 1. Now 



u(t) — u(t — e) 



1 as e 



0, 



es 



and accordingly one defines: 

(17) L[8(t)] = 1. 
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The unit impulse function at t = c is defined as 8(t — c) and one finds 

(18) L[8(t -c)] = e~ cs . 

It should be noted that L[u(t)] = L[8(t)]/s, so that by eq. (7) u(t)' can 

be thought of as an integral of 8(t) : u(t) = \ 8(t) dt. This in turn suggests 

Jo 
interpretation of 8(t) as u'(t). 

4. CONVOLUTION 

Let f(t) and g(t) be piecewise continuous for t ^ 0. Then the (Laplace) 
convolution of / and g is defined as 

(19) f*g=f f(u)g(t -u)du = h(t). 

Jo 

It can be verified that h(t) is continuous for t ^ 0, also that 

(20) hit) = f g(u)f(t - u)du = g *f. 

If now, for some a, I \f(t) | e~ ai dt and I | g(t) \ e~ at dt exist, then 
Jo Jo 



f \h(t) 



e at dt exists, so that L[f], L[g], L[h] exist and 



(21) L[h] = L[f*g} = L[f]L[g]. 
Properties of the Convolution. These are: 

(22) f*(g + h) =f*g+f*h; 

(23) /* (eg) = (cf) *g = c(f*g),c = const.; 

(24) f*(g*h) = (f*g)*h. 

Special Convolutions. The following are useful: 

(25) e at *e at = te at ; 

t n-i e at 

(26) e ai * e at * • • • * e at = (n factors) ; 

(n — 1)! 

m\n\ 

(27) t m e ai * t n e at = « OT + n + 1 e°'; 

(m + n + 1) ! 

e at _ e bt 

(28) e at * e bt = : (a ^ b). 

a — b 
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5. INVERSION 

If L[f] = F(s), one writes/ = L~ l [F\, thereby defining the inverse Laplace 
transform. The inverse is uniquely determined; more precisely, if L[f] = 
L[g] and /, g are piecewise continuous, then f = g, except perhaps at points 
of discontinuity. 

If / = L _1 [F], then as for Fourier series and integrals (Chap. 8, Sects. 
8,9), 



If 00 , If, 

(29) f(t) = — I F(s)e st do) = lim — I F(s)e at do>, 



s = a + ioi, 



at every t for which / has left- and right-handed derivatives; in the inte- 
grals a is chosen greater than the abscissa of absolute convergence of L[f]. 
Under the conditions described in Chap. 8, Sect. 8, the integral represents 
Hj[/(fo+) +/('o — )] at each jump discontinuity t . In general f(t) is de- 
fine4 to be for t < 0, which will force a discontinuity at t = unless 
f(t) — > Oast — > 0+ ; the integral thus gives 3^/(0+) at t = 0. 

Conditions for Existence. Given F(s) as a function of the complex 
variable s, one can ask whether L~ 1 [F] exists, i.e., whether F is the Laplace 
transform of some f(t). For this to hold, F(s) must be analytic in some 
half-plane a > <r , but this alone is not sufficient. If F(s) is analytic at 
s = oo and has a zero there (Chap. 7, Sect. 5), so that 

co 

(30) fOO-L-^ M>B, 

n=l S 

then F(s) is a Laplace transform: 

(31) ir 1 ^*)] = f(t) = £ a n+1 1- ; 

n=o n! 

/(£) is of exponential type and is an entire function of t (Chap. 7, Sect. 4). 
Furthermore, 

(32) , Rt) = -^ fe st F(s) ds, 

Zirl qJ 

where C is a circle: \s\ = jft > -K- If in addition, F(s) is analytic for all 
finite s except at s 1} • ■ • , s n , then f(t) equals the sum of the residues of 
F(s)e at at si, • • •, s„ (Chap. 7, Sect. 5). 

More general conditions that F(s) be a transform can be given. If, for 
example, F(s) is analytic for o- > <r ^ and is representable in the form 

(33) F(s) = - + -£4 (5 > 0), 

o o 
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where | n(s) | is bounded for a ^ a\ > o- , then F(s) is the Laplace trans- 
form of f(t), where /(0 is given by eqs. (29), with a = <r\. 

If F(s) is a proper rational function of s: F(s) = P(s)/Q(s), then eq. (32) 
is applicable and the integral can be computed by residues. If in particular 
Q(s) has only simple roots si, • • •, s n , then by Chap. 7, Sect. 5, e st P/Q has 
residue exp (skt)P(sk)/Q'(s k ) at Sk, so that 



(34) L 



— i 



.Q(«)J 






This corresponds to the Heaviside expansion formula (Chap. 8, Sect. 1). 

Particular inverse transforms can be read off Table 1 (Sect. 1) or the 
accompanying Table 2. Others can be deduced from these by linearity 
and the various rules such as (4)-(7), (9), and with the aid of convolutions. 
Extensive lists are given in Refs. 1, 5, 6 of Chap. 8. 

Rules for Finding Laplace Transforms and Their Inverses. If 
f(t) has period T, then 



1 + e~ sT J 



(35) L[f] = _ srr | e~ st f(t) dt 

1 + e 

For general /(0 with transform F(s), 

(36) L[f(at)]=-F[-), a>0; 

a \a/ 

(37) L[e- at f] = F(s + a); 

(38) L[t n f] = (-l) n F M (s), n = 1, 2, • • • ; 

(39) L[t~ n f] = f f F(s) ds---ds (n = 1, 2, • • ■). 



6. APPLICATION TO DIFFERENTIAL EQUATIONS 

Characteristic Function. Let ao, ■ • • , a n be constants, with ao ^ 0. 
The function V(s) = a s n +■••+««. will be termed the characteristic 
function associated with the differential equation 

d n x dx 

(40) a — H h a n _i — + a n x = /(<). 

di n dt 

Transfer Function. The function 

(41) Y(s) = 



V(s) a s n H h a n 

will be termed the transfer function. 
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Solutions. Let f(t) be piecewise continuous for t ^ and have an 
absolutely convergent Laplace transform for a- > <r . A solution x(t) of 
eq. (40) satisfying given initial conditions 

(42) x(0) = a , x'(0) = a h • • •, ^"""(O) = a n _x 

is obtained as follows. One forms the Laplace transform of both sides of 
eq. (40), applies the rule (6), and obtains the transformed equation 

(43) V(s)X(s) - Q(s) = F(s), 
where X = L[x], F = L[f] and 

(44) Q(s) = aoaoS w_1 

+ (a ai + aia )s n ~ 2 -\ (- (a a n -i H h ct n _ia ). 

Accordingly, 

Q(s) F(s) 

(45) X(s) = -^ + -^ = Y(s)Q(s) + Y(s)F(s), 

V(s) V(s) 

(46) x(t) = L- 1 [Y(s)Q(s) + Y(s)F(s)] = L- l [Y(s)Q(s)} + L- 1 [Y(s)F(s)]. 

Since F(s)Q(s) is a proper rational function, its inverse can be found by 
residues as in Chap. 9, Sect. 5. The inverse of Y(s)F(s) can be found in a 
variety of ways. In particular, Y(s) has an inverse transform y(t) and 

(47) L- 1 [Y(s)F(s)] = y(t) *f(t) = f y(u)f(t - u) du. 

Thus both terms in eqs. (46) are well defined and it can be shown that 
x(t) is the solution sought; x{t) has continuous derivatives through the 
(n — l)st order and an nth derivative which is continuous except where 
f(t) is discontinuous. 

The formula (47) defines y * f if f(t) is piecewise continuous for t ^ 0, 
even though f(t) may grow very rapidly as t — > +°o. If V(s) has only 
simple roots Si, • • •, s n , so that 

n A n 

(48) l / (s) = E-^~' V® = HW, 

y=i s Sj j—i 

then 



(49) 



n pt 

L~ l [YF] = 2 AjeW I f(u)e- h ) u du. 



If V has multiple roots, each multiple root sy gives rise to terms of form 
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Table 2. Inverse Laplace Transforms 
F(s) L-i[F(s)] = f(t) 

1 _ ° e (.-bla)t 

as + b a 

ps + q (q - pa)e- at - (q - pB)e~^ 

(s + a)(s + 8) 8-a ' " ^ P 

4 a P f t g , ' & 2 ~ 4ac > - - [(? - pa)e-°" - (q - pB)e-% 
as z + os + c /i 

5 Pf + g , 6 2 - 4 ac < e(-»/*» I"? cos f < + -^^ sin £ t] . 
as* + os + c La 2a ap. 2a J 

H = \/Aac — b 2 
ps 2 + qs + r - 1 [4(pa2 _ ^ + r)e _ a< 



(s + a)(s + j8)(« + 7) ABC 

a, j8, t distinct + £(p0 2 - «8 + r)e- flt 

+ C(py 2 -qy + r)e-y% 
A = 8 - 7, B = y - a, C = a - $ 

ps 2 + gs + r pB 2 - qp + r pt V pa 2 - qa + r 

(s + aYis + B)' a ^ P (B-a) 2 ^ L (B-a) 



(s + a) 2 (s + B) *- {B-a) 2 L (j8 - a) 

2affp + 

08 - «) 2 



pa 2 — 2a/3p + qB — r~\ _ 
+ W^aY 2 \ 6 ' 



ps 2 + qs + r 
(s + a) 3 



pe~ a< + (g — 2pa)te- at 

f 
+ (pa 2 — ga + r) — e~ at 



10 



LAPLACE TRANSFORMS 9-13 

Table 2. Inverse Laplace Transforms (Continued) 

F(s) L-TO] = fit) 

ps 2 + qs + r M _ at , 1 r _ t r Bs + C 



e 



+^Q 



(s + a)(as 2 + 6s + c) JV ' JV Las 2 + bs + cJ 

aa 2 — ba -\- c 7* M = pa 2 — qa -\- r, N = aa 2 — ba + c, 

B = (aq — 6p)a + V c ~ ar > 
C = (ar — pc)a + qc — br 

ps 3 + qs 2 + rs + u r p s + a "I . T _, T Pis + <7i 



Las 2 + 6s + cJ ^ L 



common roots 



(as 2 + bs + c)(As 2 + 5s + C) Las 2 + 6s + cJ LAs 2 + Bs + C. 

as 2 + bs -f- c and To find po, ?o, Pi, ?i, compute: 

^2 + Bs + C having no X() = a(ar _ c?)) + &( ^ _ ^ 

/xo = a(aw — ca) + 6cp, 

(To = a(6a — cr) + c 2 p, (5 = aB — bA 

y = aC — cA, 5o = aY 2 — 6/37 + c/3 2 , 

\i = A(Ar - Cp) + 5(5p - Aa), 

jui = A(Au - Cq) + BCp, 

o-i = A(£w - Cr) + C 2 p, 

5i = Ay 2 - 5/37 + C/3 2 . 

Then 

Xo7 — Mo/3 Mo7 — a-o/3 



Po = -z ' qo = . 

do oo 

— \i7 + Mu8 — MiT + <ri/3 

Pi = ^ • <?i = ~ 

di di 

11 (ag 2 + ^ + c) 2 ' b " - 4ac < ^ [p/3 2 * sin /3t + (a - ap)(sin # - # cos #)] 

6 n \/Aac — b 2 
2a M 2a 
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A(s — Sj)~ k in Y(s). The corresponding term in y(t) is At k ~ 1 e s J t /(k — 1)! 
and in L~ l [YF] is 

A c l 

(50) e^l f(u)(t - u) k ~ l e- s i u du. 

(k- 1)! V A 

Particular Solutions. If all the initial constants a , • • • , <x n — i are 0, 
then Q(s) = and £ = L~ l [YF] is the solution sought. This particular 
solution can be found by eqs. (47), which requires knowledge of y(t) and 
hence of the roots of V(s). This can cause difficulty. An alternative is to 
employ eqs. (29) : 

1 r 00 

(51) x(t) = — I Y(s)F(s)e st da; <r = const. > c . 

It may be possible to simplify this by residues or series expansions. 

If f(t) is of form e bt p(t), where p(t) is a polynomial of degree m in t, a 
particular solution can be found explicitly without finding the roots of V(s). 
If V(b) 9^ 0, the particular solution is 



(52) x(t) = e 



— a *>t 



Y'(b) 
Y(b)p(t) + -^- V '(t) 



+ ~ir v"(t) + • • • + — r 1 v {m \t) 

2! ml 

If 7(6) = 0, then V(s) = (s - b) k W(s), W(b) ^ 0. Let Z(s) = 1/W(s) 
and let pi (t) be the polynomial obtained by integrating 

(53) Z(6)p(0 + Z\b)p'{t) +•■■+ ii p<»>(0 

w! ' 

fc times from to t. Then x = e ht pi{t) is a particular solution of eq. (40). 
In both cases it can be verified that 

L[x] = Y(s)L[e bt p] + Y(s)R(s), 

where R is a polynomial of degree less than that of V (in fact less than that 
of W(s) in the second case). 

Simultaneous Equations. Similar methods are employed for simul- 
taneous linear differential equations with constant coefficients in unknowns 
xi, x 2 , • • • . One applies the Laplace transformation to the equations, 
thereby obtaining equations for Xi(s), X 2 (s), • • • ; in forming these new 
equations, certain initial conditions for X\, x 2 , • • • are assumed. The equa- 
tions are simultaneous algebraic equations for Xi(s), X 2 (s), • • • and can 
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be solved by elimination or determinants. When Xj(s) is known, Xj(t) can 
be found by forming the inverse transforms. 
Example. 

d 2 x dy dx d 2 y dy 

— +2- + s/= 13e 2 S _-2z4- — + 3- + 5y= 15 e 2 <; 

dr dt dt dt 2 dt 

when t == 0, x = 1, y = 0, dx/dt = 0, d?//c^ = 1. Hence 

13 

s 2 X(s) 4- (2s 4- l)7(s) = s + 



s - 2 

15 



(s - 2)X(s) 4- (s 2 4- 3s + 5)F(s) = 2 + 



X = 



s - 2 

s 4 4- s 3 4- 8s 2 + 5s + 54 



Y = 



(s-2)(s+l)(s + 2)(s 2 +l) 

s 3 + 15s 2 - 17s + 26 
(s-2)(s+l)(s + 2)(s 2 + l) 



2 19 21 43s - 51 

X = r-rr— 7 + r— ~ + 



s - 2 2(s + 1) 5(s + 2) 10(s 2 + 1) 

1 19 28 29s + 7 

Y = --——— + ——— + 



s - 2 2(s + 1) 5(s + 2) 10(s 2 + 1) 

19 , 21 , 43 cos t - 51 sin t 

x = 2e 2t e~ l + — e~ 2t -\ , 

2 5 10 

19 , 28 „, 29 cos t + 7 sin t 
y 2 5 10 

7. RESPONSE TO IMPULSE FUNCTIONS 

For many applications it is important to consider the response of a linear 
system to the impulse function 8(f) or to other ideal functions such as 

*'(«), *"(0, •••• 

Example 1. Consider the equation 

dx 

— + x = 5(0, x(0) = a . 

dt 

If one applies the Laplace transform mechanically to both sides and em- 
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ploys the rule: L[8(t)] = 1 (Sect. 3), one finds: 
1 + a 



X(s) = 



s+ 1 



x(t) = (1 + a )e- 



Hence x(t) has a discontinuity at t = (Fig. 6) ; x jumps from the assigned 
initial value a to the value 1 + a . 
Example 2. Similarly, 

d 2 x dx 
is found to have the solution (Fig. 7) x = 1 + a + a\ — (1 -f- ai)e~ i , with 

1 + «o 




Fig. 6. Response of first order system Fig. 7. Response of second order sys- 

to 6-function. tern to 5-function. 

a = x(0), ax = x'(0). Here there is no discontinuity of x(t) at t = 0, but 
x'(t) has a discontinuity, jumping from the assigned initial slope of «i to 
the slope 1 + «i. 

It should be noted that the second example can be written as follows: 



dx 
dt 



= y, 



dy 
dt 



thus its solution is an integral of the solution of the first example. Each 
such integration reduces the type of discontinuity. In general, 



V(D)x = 8(f), 



V(D) = a D n + 



a ?* 0, 



has a solution which has a jump in the (n — l)st derivative at t = 0, but 
no jumps in the derivatives of lower order. For the equation 

V(D)x = 8(t - c), c > 0, 

a similar conclusion holds, with the discontinuity occurring at t = c. 
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One can interpret 8(t — c) as — u(t — c), if one forms the transforms by 

the rule L[f] = sL[f], ignoring the discontinuity which would make the 
rule inapplicable. For then 



d 

— u(t — c) 
Idt 



= sL[u{t — c)] = e 



in accordance with eq. (18). This suggests the general procedure. 
General Procedure. For the differential equation 

df 
V(D)x = f , 
at 

in which/ has a jump discontinuity at t = c but has otherwise continuous 
derivatives, one should take transforms ignoring the discontinuity: 

V(s)L[x] =sL[f]-f(0). 

Under similar conditions on /, a similar procedure can be used for higher 
derivatives, and for the general equation 

V(D)x = W(D)f. 

If the order of V(D) is less than that of W(D), x will itself be an ideal 
function ; otherwise x will merely show some discontinuity at t = c. Similar 
remarks apply when there are several jump discontinuities. 

Let / have continuous derivatives of all orders except for t = c, at which 
the derivatives have limiting values to the left and to the right. Then / 
can be written as/i(f) + k x u(t — c), where f x (t) is continuous at t = c; cor- 
respondingly, f(t) = f'\{t) + kib(t — c), where f\(t) is discontinuous at 
t = c. Thus 

f(0 = / 2 (0 + hu(t - c) + hd(t - c) 

f"(t) = f' 2 (t) + k 2 d(t - c) + hd'it - c) 

= M) + k 3 u(t - c) + k 2 8(t - c) + M'(* ~ c), 



Computation of L\S'], L[f"], • • •, as described above, is then equivalent to 
that obtained by writing 

L[f] = L[f 2 ) + k 2 L[u(t - c)] + hL[8(t - c)] 

L[f] = L[h\ + k 3 L[u(t - c)] + k 2 L[8(t - c)] + hL[8'(t - c)] 
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if one agrees that 

(54) L[b {m \t - c)] = s m e- cs (w = 1, 2, • • •)• 

The justification for the rules adopted lies in the fact that they give a 
reasonable limiting form for the response x(t), and they meet the needs of 
the physical situations to which they are applied. 

8. EQUATIONS CONTAINING INTEGRALS 

The method of Sect. 6 is applicable to "integro-differential equations" 
such as the following : 

dx r f 

(55) a \- ayx + 0,2] xdt = f(t) . 

dt Jq 

One need only apply the Laplace transformation to both sides and employ 
rule (7): 

X 

(56) a (sX - oq) + a x X + a 2 - = F(s), 

s 

from which one can solve as before for X(s). 

One can also differentiate eq. (55) to obtain an equation of second order: 

d % dx 

(57) a — + ai — + a 2 x = f'(t) ; 

dt z dt 

from eq. (55), a x'(0) + aix(0) = /(0), so that one initial condition for eq. 

(57) is fixed. If f(t) has discontinuities, f'(t) has to be treated as an ideal 
function (Sect. 7); in such a case, it is simpler to use eq. (56). 

It should be remarked that eq. (55) is equivalent to the system 

dx dv 

«o — + «iz + a 2 y = /(<), — = x, 

dt dt 

with the initial conditions: x(0) = a , 2/(0) = 0. By similar devices inte- 
grals can be eliminated formally in most cases. 

9. WEIGHTING FUNCTION 

It has been seen that, for proper initial conditions, various problems lead 
to relations of form 

(58) X(s) = Y(s)F(s), 

where F(s) is the Laplace transform of a driving function or "input" f(t) 
and X(s) is the Laplace transform of the "output" x(t). In such cases 
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Y(s) is termed the transfer function; i.e., in general, the transfer function 
is the ratio of the Laplace transforms of output and input. 

If y(t) is the inverse Laplace transform of Y(s), then as in Sect. 6 



(59) 



x{t) =y(t)*f(t) 



= ff(t- 
Jo 



u)y(u) du. 



Accordingly, x(t) is a weighted average of f(t) over the interval from to t, 
the value at t — u receiving weight y{u). Since f(t) = for t < 0, one can 
also write 

(60) x( 



*(0 = I f(t — u )y( u ) du, 

J rr, 



so that the average is over the entire "past" of /(£). 

Graphical Computation. One can then compute x{t) at each t 
graphically as suggested in Fig. 8. Here y{u) is graphed against u, with 




Fig. 8. Response as a weighted average. 

the positive w-axis to the left and the origin above the point t on the i-axis. 
The value of / at t — u is multiplied by the value of y above t — u and the 
result is integrated to yield x{t) at the t chosen. As the graph of y{u) is 
moved parallel to the f-axis, the average at successive times t can be found. 
Weighting Function. The function y(t) = L~ 1 [Y(s)] is termed the 
"weighting function." In view of the discussion given, this term would be 



justified only if | y(t) dt = 

Y(s) = f y(t)< 
Jo 



(61) 



1. But 



dt. 



Y(0) 



J o 



y(t) dt, 



provided Y(s) is denned for s = 0. Hence if F(0) = 1, the "total weight" 
is 1, as desired. If F(0) ^ oo, one can redefine the input as a constant 
times x(t) and achieve the same result. 
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Response to Unit Impulse. If e is very small and f(t) is a'square pulse 
of height 1/e from t = to t = e, then eqs. (59) show that approximately 

x(t) =-y(t)-e = y(t); 

€ 

as e — > 0, this can be shown to be the limiting relation. Thus the weighting 
function is the response to the unit impulse function 8(t). This also follows 
from eq. (58), since if f(t) = 5(0, L[f] = F(s) = 1. 

One can also remark that, if /(0 is the unit function u(t), then L[f] = 
F(s) = 1/s, so that by eq. (58) 

Y(s) dx 

X(s) = -1^, Y(s) = sX(s), y(t) = - ; 

s dt 

for, by eqs. (59), x(0) = 0. Thus the weighting function can be interpreted 
as the derivative of the response to the unit function. If one denotes by A (t) 
the response to the unit function, so that L[A] = Y(s)/s, then for an arbi- 
trary driving function /(i), 



(62) 



X(s) = s ( -^ F(s) ), x = - I f(t- u)A(u) du. 

\ s / dtJ 



Equations (59) and (62) are equivalent to the eqs. (25), (26) of Chap. 8, 
Sect. 5. 

10. DIFFERENCE-DIFFERENTIAL EQUATIONS 

Because of the transformation rule: L[f(t — c)] = e~ cs L[f] (Sect. 3), 
Laplace transforms can be applied to solve linear difference-differential 
equations, i.e., equations of form 

n M 

(63) £ ILci mk f k \t-mT) =g(t); 

k=0m=0 

it will be assumed that the coefficients a m ^ are constants and that a solu- 
tion f(t) is to be found which is equal to for t ^ and satisfies eq. (63) for 
t > 0. Under these conditions 

(64) L[f h \t - mT)\ = s k L[f(t - mT)] = s k e- mTs L[f] 
and the transformed equation corresponding to eq. (63) is 

/ n M \ 

(65) ( £ £ a mk s k e- mTs ) F(s) = 0(a). 
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This can be solved for F(s) and the solution sought is L~ 1 [F(s)]. Validity 
of this process requires in particular that for some a the term in paren- 
theses in eq. (65) have no zeros in the complex s-plane for o- > cr . For 
discussion of the questions involved here see Ref. 1. 

Instead of requiring that fit) be = for t < one can impose the condi- 
tion that fit) coincide with a given function f (t) in an "initial interval" 
— MT ^ t ^ 0. This case can be reduced to the previous one by first ex- 
tending the definition of f (t) to the range t > 0, while preserving con- 
tinuity, and introducing a new unknown function /i (t) = fit) — fo(t). 

11. ASYMPTOTIC BEHAVIOR OF TRANSFORMS 

In general the behavior of fit) at t = is related to that of F(s) = L[f] 
as s — > oo along the real axis, while the behavior of f(t) at t = +oo is re- 
lated to that of F(s) as s — > (or s — * <r ) along the real axis. A full dis- 
cussion is given in the book of Doetsch (Ref. 3, Chap. 8), pp. 186-277. 

If 

a x G(s) 

(66) F(s) = - + -y., 

where | G(s) \ < M f or a > cro, then . 

(67) lim f(t) = lim sF(s) (s real). 

If f(t) and f'(t) have convergent Laplace transforms for a > and fit) 
has a limit as t — > -f-w, then 

(68) lim fit) = Mm sF is) (sreal). 

t—f CO S— >0 
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1. DEFINITION OF CONFORMAL MAPPING. GENERAL PROPERTIES 

Definitions. Let u = fix, y), v = g(x, y) be two real functions of the 
real variables x, y, both defined in an open region D of the xy-pl&ne. As 
(x, y) varies in D (Fig. 1), the corresponding point (u, v) varies in a set 
D\ and one says that the equations 

(1) u = f{x, y), v = gix, y) 

define a transformation or mapping T of D onto Z>i (Chap. 1, Sect. 3). If 
for each (w, v) in D± there is precisely one ix, y) in D such that u = fix, y), 
v = gix, y), then the transformation T is said to be one-to-one, and T has 
an inverse T~ l , defined by equations 

(2) x = <f>iu, v), y = ypiu, v), 

obtained by solving eqs. (1) for x and y in terms of u and v. 

Now let T, defined by eqs. (1), be a mapping of D onto D\. In addition, 
let fix, y) and gix, y) have continuous first partial derivatives in D. The 
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mapping T is said to be conformed if, for each pair of curves C\, C 2 meeting 
at a point (x 0} y ) of D, the corresponding curves C\, C 2 meeting at (u , v ) 
form an angle a at (w , v ) equal to that formed by C{*, C 2 * at (x , y ). It 
is assumed that C\, C 2 are directed curves and have well-defined tangent 
vectors at (x , yo) so that C\*, C 2 * also have tangent vectors at (u , v ). The 
angle a is then measured between the tangent vectors. It is customarily 
a signed angle and measured, e.g., from C\ to C 2 and, correspondingly, from 





(a) (b) 

Fig. 1. Conformal mapping: (a) s-plane, (b) w-plane. 



, 2 - . Conformality then means that the corresponding angles are 
equal and have the same sense, as in Fig. 1. To emphasize this, one can 
write more explicitly that T is to be conformal and sense-preserving. For 
most applications T is assumed to be one-to-one. Conformality of T then 
implies conformality of T -1 . 

Theorem 1. Let (1) define a mapping T of D onto D x . Let f(x, y) and 
g(x, y) have continuous first partial derivatives in D. Then T is conformal 
and sense-preserving if and only if the Cauchy-Riemann equations 



(3) 



du 
dx 



dv 
dy 



du 
dy 



dv 
dx 



hold in D and the Jacobian d(u, v)/d(x, y) 9^ in D. 

By virtue of this theorem, the theory of conformal mapping is related 
to the theory of analytic functions of a complex variable (Chap. 7). One 
can use complex notation: 



(4) 



z = x + iy, 



w = u + iv, 



and the transformation T is then simply a complex function w = F(z) de- 
fined in D. The mapping w = F(z) is conformal precisely when F is analytic 
in D and F'{z) ^ in D. (See Ref. 2.) 
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Remark. If F'{z) is at a point z , then z is termed a critical point of 
F(z). A function w = F(z) cannot define a conformal mapping of any open 
region D containing a critical point z . The behavior of F(z) near a critical 
point is typified by the behavior of z n near z = 0, for n = 2, 3, • • • ; except 
for w = 0, each w has n inverse values w lln . Curves meeting at angle a at 
z = are transformed onto curves meeting at angle na at w = 0. The 
absence of critical points does not guarantee that F(z) describes a one-to- 
one mapping; all that can be said is that, if F'(z ) j± 0, then w = F(z) does 
define a one-to-one conformal mapping of some sufficiently small region 
containing z . 

Geometrical Meaning of Conformality. Let w = F(z) define a 
one-to-one conformal mapping of D on D\. Then each geometrical figure 





H' = F' 



H E 

Fig. 2. Behavior of mapping in the interior and on the boundary. 



in D will correspond to one in Di which is similar in a certain sense; if the 
first figure is bounded by smooth arcs, the second will be bounded by similar 
arcs and corresponding pairs of arcs form the same angle (Fig. 2). The 
lines x = const., y = const, in D form two families of curves meeting at 
right angles; hence these correspond to curves in D\ formed of one family 
and of its family of orthogonal trajectories (Fig. 3). Similarly the curves 
u = const, form orthogonal trajectories of the curves v = const. On the 
boundary of D conformality may break down. In general there is some 
sort of continuous correspondence between boundary points of D and those 
of Z>i. If D and D\ are each bounded by several simple closed curves, and 
F is one-to-one, then the mapping F and its inverse can indeed be extended 
continuously to the boundaries. Commonly there are points at which con- 
formality is violated in that two boundary arcs of D meeting at angle a 
correspond to boundary arcs of D\ meeting at angle /3 ^ cc; in particular 
this can mean a folding together of the boundary, as suggested in Fig. 2. 
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As in Chap. 7, Sect. 5, one can adjoin the number oo to the complex 
plane to form the extended plane. The mapping w = F(z) is said to be 
conformal in a region containing z = <x> if F(l/z) is conformal in a region 
containing z = 0. Similarly, one can discuss conformality in a neighbor- 
hood of a point z at which F(z Q ) — oo, so that F(z) has a pole, in terms of 
the conformality of 1/F(z) near z . 




(a) (6) 

Fig. 3. Level curves of x and y: (a) z-plane, (b) w-plane. 



Conformal Equivalence. Two regions D, Di are said to be conformally 
equivalent if there is a one-to-one conformal mapping w = F(z) of D on D x 
(so that the inverse function maps D x conformally on D). Conformally 
equivalent regions must have the same connectivity; i.e., if D is simply 
connected, then so is D\) if D is doubly connected, so is D\. However, 
having the same connectivity does not guarantee conformal equivalence. 
If D is simply connected then D is conformally equivalent to one and only 
one of the following three: (a) the interior of a circle; (6) the finite plane; 
(c) the extended plane. In particular, one has the following theorem. 

Theorem 2 (Riemann Mapping Theorem). Let D be a simply con- 
nected region of the finite z-plane, not the whole finite plane. Let z be a point 
of D, and let a be a given real number. Then there exists a one-to-one conformal 
mapping w = F(z) of D onto the circle \w\ < 1 such that F(z ) = and 
org F'{zq) = a. Furthermore, F{z) is uniquely determined. 

From this theorem it follows that the one-to-one conformal transforma- 
tions of D onto \w\ < 1 depend on three real parameters: x = Re (z ), 
yo = Im (z ) and a. These parameters can be chosen in other ways. For 
example, three boundary points of D can be made to correspond to 3 points 
on \w\ = 1 (in the same "cyclic order"). 



CONFORMAL MAPPING 



10-05 



2. LINEAR FRACTIONAL TRANSFORMATIONS 
Each function 



(5) 



w = 



az + b 

1 

cz + d 



a b 
c d 



*0, 



where a, b, c, d are complex constants, defines a linear fractional transfor- 
mation. Each such transformation is a one-to-one conformal mapping of 
the extended z-plane onto the extended w-plane. Special cases of eqs. (5) 
are the following: 

Translations. The general form is 



(6) 



w = z + b. 



Each point z is displaced through the vector b. 
Rotation Stretchings. The general form is 



(7) 



w = az = Ae la z. 



The value of w is obtained by rotating z about the origin through angle a 
and then increasing or decreasing the 
distance from the origin in the ratio 
A to 1. 

Linear Integral Transformations. 



Complex plane 



(8) 



w = az + b. 



Each transformation (8) is equivalent 
to a rotation stretching followed by a 
translation. 

Reciprocal Transformation. 



(9) 



1 



w = 




Fig. 4. The transformation w = 1/z. 
Here \w\= 1/ 1 z \ and arg w = — arg z. 

Hence w is obtained from z by "inversion" in the circle \z\ = 1 followed 
by reflection in the x-axis (Fig. 4). 

Important Conformal Mappings. The general linear fractional trans- 
formation (5) can be composed of a succession of transformations of the 
special types: 



(10) 



w 



a be — 

- + 

c c 



ad 



r, 



f = 



Z = cz + d. 
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If one includes straight lines as "circles through =°," then each transforma- 
tion (5) maps each circle onto a circle. By considering special regions 
bounded by circles and lines one obtains a variety of important conformal 
mappings, as illustrated in Table 1. The first three entries in the table 
depend on 3 real parameters and provide all conformal mappings of D on 
Di in each case.. 

.,. Table 1. Important Conformal Mappings 
F(z) D D 1 

|z|<l |H<1 



z — Z 



1 — ZoZ 
a real, | zo | < 1 

az + b 
cz + d 

a, b, c, d real, ad — be > 

z — Zo 

e ia -> 

z — Zo 

a real, Im (zo) > 



Im (z) > Im (w) > 



lm(z)>0 \w\<l 



region between circles — < Re (w) < — 

z , , 2b 2a 

\z — a\— a, 

■ \z-b\=b, 
< a < b 

To find a one-to-one conformal transformation of a circular region D on 
a circular region D\ f one can choose three points z±, z 2 , z 3 on the bounding 
circumference of D and three points w\, w 2 , w 3 on the bounding circum- 
ference of Di, in the same cyclic order as z x , z 2 , z 3 . The equation 

W — Wo Wi — Wo Z — Zo Z\ — Zo 

(H) + = -=. 

w — W 3 Wt — w 3 z — z 3 z 1 — z 3 

then defines a linear fractional transformation w = F(z), mapping D on 
Di, and moreover F(z } ) = Wj for j = 1, 2, 3. The left-hand side of eq. (11) 
is an expression formed from four complex numbers w, w i} w 2 , w 3 , termed 
the cross-ratio of the four numbers in the order given, and denoted by 
[w, Wi, w 2 , w 3 ]. Equation (11) states that [w, w\, w 2 , w 3 ] = [z, Z\, z 2 , z 3 ], 
or that the cross-ratio is invariant for a linear fractional transformation. 

3. MAPPING BY ELEMENTARY FUNCTIONS 

Except for the linear fractional transformations, the elementary analytic 
functions define one-to-one conformal transformations only in suitably re- 
stricted regions. 
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The Function iv = s 2 . This maps both z and —z on the same w, and 
hence it is one-to-one only if restricted to a region D which contains no 
pair z, —z. For example, D can be chosen as the upper half-plane Im (z) 
> 0. The corresponding region D\ consists of the w-plane minus the ray: 
u ^ 0, v = 0. The points (x, 0) on the boundary of D correspond to the 
points (u, 0) on the boundary of D i} both (x, 0) and (— x, 0) corresponding 
to (u, 0), with u = x 2 . It should be noted that F'(z) = 2z is at z = 0, 
so that this point is critical; conformality fails here, and in fact the edges 
of D, forming a 180° angle at z = 0, are transformed onto overlapping 
edges of Di which form a 360° angle. 

For w = z 2 one can also choose D as a sector a < arg z < /3, provided 
/3 — a < 7r; the region D\ is the sector: 2a < arg w < 2/3. A third choice 
of D is a hyperbolic region: xy > 1, x > 0; D^ is then a half-plane, v > x /i. 
A fourth choice of D is a strip: a < x < b, where a > 0; D x is then a region 
bounded by two parabolas: 4a 2 w -f- v 2 = 4a 4 , 4b 2 u -\- v 2 = 4b 4 . 

The Function iv = s 11 . Analogous choices of regions can be made 
for w = z n (n = 2, 3, 4, • • •)• The sector D: a < arg z < /3, with (3 — a 
< 2ir/n, corresponds to the sector Dx : na < arg io < n/3. If n is allowed 
to be fractional or irrational, w = z n becomes a multiple-valued analytic 
function (Chap. 7, Sects. 6 and 7) and one must select analytic branches. 
For such a branch the mapping of sectors is similar to that when n is an 
integer. 

The General Polynomial w = aoZ n + • • • -f- a n _is + a n . Suitable 
regions can be obtained by means of the level curves of u = Re (w) and 
v = Im (w). In particular the level 
curves of u and v which pass through ^ y * 
the critical points of F{z) divide the 
z-plane into open regions each of which 
is mapped in one-to-one fashion on a 
region of the w-plane. This is illus- 
trated in Fig. 5 for w = z 3 — 3z + 3. 

The critical points are at z = ±1, at 

which v = 0. The level curve v = in / IV 

divides the z-plane into six regions, in 

each of which w = F{z) describes a 

one-to-one conformal mapping of the 

region onto a half-plane. Adjacent 

regions, such as_I and IV, can be FlG 5 Mapping by w = z z _ 3z + 3 . 

merged along their common boundary 

to yield a region mapped by w = F{z) on the w-plane minus a single line. 

The Exponential Function w = e 3 . This maps each infinite strip 
a < y < b conformally onto a sector a < arg w < b, provided b — a ^ 2t; 
in particular each rectangle : c < x < d, a < y < bin the strip corresponds 



VI 
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to the part of the sector lying between the circles \w\ = e c and \w\= e d . 
Similarly, the inverse of the exponential function, w = log z, maps a sector 
on an infinite strip. When b — a = v/2, the sector is a quadrant; when 
b — a = it, the sector is a half-plane. 

The Trigonometric Function w = sin z. This maps the infinite 
strip — 7r/2 < x < 7r/2 on the finite w-plane minus the portion |Re (w) | 
^ 1 of the real axis. 

The Rational Function w = z + (1/s) = (z 2 -f- l)/z. This maps the 
exterior of the circle \z\= 1 on the w-plane minus a slit from — 2 to +2. 
The same function maps the upper half-plane Im (z) > on the w-plane 
minus the portion | Re (w) | ^ 2 of the real axis. 

Let the real constants hi, • ■ •, h n+1 , xi, • • •, x n satisfy the conditions 

hi < h<2, < - • • < h m , h m > h m+ i > • • • > h n+ i, 
(12) 

Xi < x 2 < • • • < x n , 

for some m, 1 ^ m ^ n + 1. Then 

A 2 — Zfc 

(13) /(z) = /ii log (z - x x ) - h n+ i log (z - z n ) + 2^ h log 



fc=2 2 — X k _i 

maps the half-plane Im (z) > one-to-one conformally on a region D x con- 
sisting of a strip between two lines v = const, minus several rays of form 
v = const. If the strip Di has width ^ 2ir, the function F(z) = exp [f(z)] 
maps the upper half-plane conformally and one-to-one on a sector minus 
certain rays and segments on which arg w = const. (See Chap. 7, Ref . 7, 
pp. 605-606.) 

4. SCHWARZ-CHRISTOFFEL MAPPINGS 

These are defined by the equation 

r z dz 
(14) w =/(z) = A - + B, 

Jxo (Z ~ Xx) fc i • • • (Z - XnYn 

where A, B are complex constants, x , x\ } • • •, x n , k i} • • •, h n are real con- 
stants, and — 1 ^ kj ■ ^ 1. The function /(z) is analytic for Im (z) > 0, 
with (z — Xj) kJ interpreted as the principal value: exp [kj log (z — Xj)]. 
Every one-to-one conformal mapping of the half-plane D onto the interior 
of a polygon can be represented in the form (14) ; this applies more generally 
to every one-to-one conformal mapping of the half-plane onto a simply 
connected region whose boundary consists of a finite number of lines, line 
segments, and rays. 

Polygon. When the function maps D onto a polygon, the points Xi, 
• • • , x n (and possibly °o) on the z-axis correspond to vertices of the polygon, 
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and the corresponding exterior angles are A471-, • ■ •, k n T. If there is an 
(n + l)st vertex, corresponding to z = <», then necessarily k\ + • • • + Ic n 

^ 2; in general, 1 < k\ -\ 1- k n < 3. 

Convex Polygon. When the function (14) maps D onto a convex poly- 
gon, all exterior angles are between and ir and the sum of the exterior 
angles is 2w; accordingly, 

(15) < kj < 1 and k x + • • • + k n ^ 2. 

When k\ -\ \- k n < 2, there is an (n + l)st vertex corresponding to 

z — 00. In general, for every choice of the numbers k\, • - • , k n such that 
(15) holds, eq. (14) describes a one-to-one conformal mapping of the 
half-plane Im (z) > onto the interior of a convex polygon. 
Rectangle. For the special case 



( 16 ) w = f —, „ ,T„ -nnK- ' < fc < 1, 



VU - z*Xi - fcV) 

the mapping is onto a rectangle with vertices ±i£, ±2£ -f- iK' , where 
dz r llk dz 



r 1 dz z* 1 

J V(l - 2 2 )(1 - ^2 ) J I 



V(i - 2 2 )(i - fcV) 



In this case jF(z) is an elliptic integral of the first kind (Chap. 7, Sect. 8), 
and its inverse is the elliptic function z = snw. 

A great variety of conformal mappings have been studied and classified. 
See Ref . 1 for an extensive survey. 

5. APPLICATION OF CONFORMAL MAPPING TO BOUNDARY VALUE 
PROBLEMS 

The applications depend primarily on the following formal rule. If 
U(x, y) is given in a region D and w = f(z) is a one-to-one conformal map- 
ping of D on a region D 1} then 



d 2 U d 2 U 



dx 2 by 2 Ldu* dv* 



d 2 U d 2 U 



+ 



l/'CO 



In particular, U is harmonic in terms of x and y\ 

d 2 U d 2 U 
~dx J ^~dy 1 

if and only if U is harmonic when expressed in terms of u and v. 



< 19 ) TT + TJ = 0, 
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The boundary value problems considered require determination of U in 
D when U is required to satisfy some conditions on the boundary of D 
and to satisfy an equation 

d 2 U d 2 U 

(20) —J + —j = h(x, y), 

dx 2 by 2 

for given h(x, y), in D. It follows from eq. (18) that a conformal mapping 
w = f(z) amounts to a change of variable reducing the problem to one of 
similar form in the region D\. It is in general simpler to solve the problem 
for a special region such as a circle or a half-plane. Hence one tries to find 
a conformal mapping of D onto such a special region D\. Once the prob- 
lem has been solved for U in D\, U can be expressed in terms of (x, y) in 
D and the problem has been solved for D. 

For most cases D has a boundary B consisting of a finite number of 
smooth closed curves C 1 , •••, C n , the case n = 1 being most common. 
The most important boundary value problems are then the following. 

I. Dirichlet Problem. The values of U on B are given; U is required 
to be harmonic in D and to approach these values as limits as z approaches 
the boundary. 

II. Neumann Problem. Again U is harmonic in D but on B the 
values of d U/dn are given, where n is an exterior normal vector on B. 

Both problems can be generalized by requiring that U satisfy a Poisson 
eq. (20) in D. In general this case can be reduced to the previous one by 
introducing a new variable W, where 

(21) w=U + ^-f fhfa v ) log [(x -0 2 + (y-r,) 2 ] ft dr,. 

Furthermore, the Neumann problem can be reduced to the Dirichlet prob- 
lem by consideration of the harmonic function V(x, y) conjugate to U 
(Chap. 7, Sect. 2). 

To solve the Dirichlet problem for a simply connected region D, one 
seeks a one-to-one conformal mapping of D on the circular region \w\ < 1. 
This reduces the problem to a Dirichlet problem for the circular region. 
If p(u, v) are the new boundary values, its solution is given by 



,2tt 



1 



i r 

(22) U = — p(cos 0, sin <£) — — dcj>, 

2irJ 1 + r 2 - 2r cos (0 - 0) 

where r, 6 are polar coordinates in the wv-plane. 
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If D is multiply connected, it is also possible to map D conformally on 
a standard type of domain, for which solution of the Dirichlet problem is 
known. For details, see Ref. 2. 
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1. TABLE OF NOTATIONS 

Table 1 lists notations in current use. There are some inconsistencies 
between different systems, and care is needed to ensure proper understand- 
ing. The list is not exhaustive and there are other notations even for the 
crucial relations; for example, "a and b" is sometimes denoted by "ab." 
The grouping under mathematics, engineering, and logic is somewhat arbi- 
trary. 

2. DEFINITIONS OF BOOLEAN ALGEBRA 

First Definition. A study of the rules governing the operations on 
sets (Chap. 1) leads to a type of algebraic system, in which the basic opera- 
tions are U and fl , frequently called "or" and "and," corresponding to 
union and intersection of sets. In addition, the system can be partially 
ordered (the relation of set inclusion) and each object of the system has a 
complement. 
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Table 1. Table of Symbols, Boolean Algebra 





Operation Name 




Symbols 












Engi- 




Mathematics 






Mathematics 


neer- 




(Set Theory) 


Engineering 


Logic 


(Set Theory) 


ing 


Logic 


Union 


"or" 


"or" 


U 


+ 


V 


Intersection 


"and" 


"and" 


n 


• 


A 


Symmetric 


Exclusive 


"or" 


©or + 


None 


A 


difference 


"or" 










Complement 


Complement 


Negation 


'orC 


— 


~ 


Order 


Order 


Material 
implication 


< 


=» 


z> 


Sheffer 


. Sheffer 


Sheffer 


1 


1 


1 


stroke 


stroke 


stroke 








Existential 


Existential 


Existential 


3 or(J 


V 


3 orZ 


quantifier 


quantifier 


quantifier 


t 


t 


t t 


Universal 


Universal 


Universal 


n 


A 


V orTJ 


quantifier 


quantifier 


quantifier 


t 


* 


t t 



A Boolean algebra B is a set of elements x, y, z, • • • with two binary opera- 
tions U and fl , an order relation S. , and operation ' of forming the comple- 
ment such that: 

(1) x U x = x, x fl x = x, 

(2) x l) y = y \J x, x D y = y D x, 

(3) x (1 (y n z) = (a; (1 y) fl z, x U (y U z) = (x U y) U z, 

(4) x (y U z) = (x D y) U (x fl z), 
x U (y PI z) = (x U y) (1 (x U 2), 

(5) a; ^ x, 

(6) x ^ y and i/ ^ 2 imply a; ^ 2, 

(7) x ^ y and y ^ x imply x = y, 

(8) 5 contains two elements and 1 such that ^ x < 1 for all x in 5, 

(9) fl x = 0, 1 fl a; = x, 

(10) U a: = a:, 1 U x = 1, 

(11) xVl x' = 0, x\J x' = 1, 

(12) (a; fl y)' = x' Uy', (x U y)' = a/ fl 1/', 

(13) (a;')' = a;. 
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The properties (1) to (13) can be regarded as a set of postulates, from 
which all other properties are to be deduced. Some of the postulates are 
consequences of others, so that the list could be considerably reduced (Refs. 
1,3,5). 

The definition given here is easily verified to be equivalent to that given 
in Chap. 1, Sect. 7, in terms of lattices (Ref. 3). 

Second Definition. An alternative definition is based upon the set 
operation of symmetric difference, also known as "exclusive or." The sym- 
metric difference of two sets X, Y, denoted by X © Y, is the set of all ele- 
ments in X, or in Y, but not in both. In symbols, 

(14) X@Y = {s\sCX UFandsCinF). 

This is pictured in Fig. 1. 

From the definition, a number of properties can be verified. For exam- 
ple, X © X = (here the empty set plays the role of the of a Boolean 




Fig. 1. Symmetric difference. Fig. 2. Three-term symmetric difference. 

algebra), (X © Y) © Z = X © (Y © Z). The proof of the second rule is 
suggested in Fig. 2. 

In an arbitrary Boolean algebra one can define x © y in terms of the 
other operations 

(15) x@y = {x 2/') U (a/ fly). 

From (1), • • •, (13) and (15) a number of rules can then be deduced by 
algebraic means alone. 

It is possible to consider © and (1 as the basic operations and express U , 
', and ^ in terms of these two: 

(16) x U y = (x © y) © (x fl y), 

(17) x' = l®x, 

(18) x ±5 y if x D y = x. 

Pursuing this point of view further, one is led to a second definition of a 
Boolean algebra. 
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Alternative Definition. A Boolean algebra B is a set of elements re, y, 
z, • ' • with two binary operations 0, fl satisfying the laws: 

(19) x@y = y@x, x fl y = y f\ x, 

(20) (x@y)@z = x®(y@z), (x fl y) D z = x D (y (1 z), 

(21) x fl (y@z) = (re fl y) © (x D z), 

(22) re Pi a; = x, 

(23) jB contains two elements and 1 such that for all x in B, x = re, 
re © re = and re fl 1 = re. 

If the rules (19) to (23) are regarded as postulates and the relations 
(16), (17), (18) as definitions of U, ', and ^, then one can prove all the 
laws (1) to (13). Conversely, from (1) to (13) and the definition (15), one 
can prove (19) to (23). Hence the two definitions of a Boolean algebra are 
equivalent. 

Relation to Set Theory. Although Boolean algebras arise naturally 
in set theory, that is not the only source of such systems. They arise in 
logic and in other mathematical contexts. It is natural to ask whether 
every Boolean algebra can be interpreted as an algebra of all subsets of a 
given set. This is not true as stated, but there is a close relationship between 
each Boolean algebra and an algebra of sets (Sect. 5) . 

Example 1. A very simple but nevertheless useful Boolean algebra is 
one in which B contains only and 1. The properties are given in Tables 
2 and 3. This Boolean algebra is used in switching circuits: x = 1 means 



Table 2. 


x U y 


Table 3 


x n y 


\ y 




\ y 




x \ 


1 


X \ 


1 



1 


1 

1 1 



1 



1 



that a certain switch is closed and re = means that the switch is open. 
Two switches in parallel correspond to re U y; two switches in series corre- 
spond to re fl y. 

Example 2. A somewhat more general Boolean algebra is used in the 
design of electronic digital computers. This can be described as follows. 
The elements of B are all ordered n-tuples re = (rci, re 2 , • • •, x n ), where 
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each Xk is or 1; if y = {y\, • • •, y n ), then x U y, x (~l y are defined as 
follows : 

x U ?/ = (zi U 7/i, rc 2 U 2/2, • • •, x n U ?/ n ), 

a; D y = (xi fl t/i, x 2 fl ?/ 2 , • • •, x n C\ y n ), 

where xu U y k , Xk H y k are evaluated as in Tables 1 and 2. The and 1 
of B are defined as follows : 

0=(0,0, ...,0), 1 = (1,1, ...,1). 

Electronic devices can be constructed to perform the Boolean operations 
on the n-tuples x. The operation of ordinary arithmetic can be defined 
in terms of the Boolean operations together with the operation of shifting 
the decimal point. 

3. BOOLEAN ALGEBRA AND LOGIC 

Algebra of Sentences. Let x, y, • • • stand for declaratory sentences. 
For example x might stand for "greed is evil" and y for ''lead is heavy." 
From two sentences x, y one can form the new sentence "x and y"; this is 
denoted by x D y. In the example given, x C\ y is the sentence "greed is 
evil and lead is heavy." From x and y one can also form the sentence 
"x or y"; this is understood to mean: x or y, but not both; the new sentence 
is denoted by x y. One can also form the statement "x and/or y," mean- 
ing: x or y or both; this is denoted by x U y. Finally, one can form the 
negation of a sentence x: "lead is heavy" when negated becomes "lead is 
not heavy." The negation of x is denoted by x'. 

One can now verify that in the normal logical procedures for manipulat- 
ing sentences, the operations U , D , 0, ' obey all the rules of a Boolean 
algebra. Two sentences are regarded as equal if they are logically equiva- 
lent. In this sense, all false sentences can be considered equal and identi- 
fied with the of the Boolean, algebra; a universal truth ("tantology") can 
serve as the 1. In logic a table showing the Boolean algebra relationship of 
variables is called a truth table. The order relation x ^ y can be inter- 
preted to mean: "x implies y." For example, if x is the sentence "t is an 
even integer" and y is the sentence "2t is an even integer," then x ^ y, 
but y ^ x is false, so that x < y. The implication defined here is essen- 
tially strict implication (see below) (Refs. 4, 5). 

Propositional Functions. The sentence "t is an even integer" con- 
tains a variable, t. Accordingly, the sentence can be regarded as a function 
of t. For each value of t, the function becomes a definite sentence or 
proposition. Hence the function is termed a propositional function. It 
can be denoted by /, with f(t) denoting the value for each t . For example, 
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/(4) is the true sentence "4 is an even integer," while /(3) is the false sen- 
tence "3 is an even integer." The fs for which f(t) is true form a set. 

Similarly, the sentence "t is a human being" is a propositional function 
which in turn determines a set; namely, the set of all human beings. If 
f, g, • • • are propositional functions, then one can form new propositional 
functions / U g, f C\ g, /© g, /', • • • as above. If Xf, X g , • • • are the sets 
corresponding to these propositional functions, then the operations on the 
functions correspond precisely to the operations U , fl , 0, ' on sets. For 
example, f D g is true when / and g are true; therefore an object belongs to 
X/ ng when it belongs to Xf and to X g , that is, to Xf D X g . Thus the cal- 
culus of propositional functions can be interpreted as a Boolean algebra of 
sets. The zero element represents a propositional function which is false 
for all values of the variable; the set 1 corresponds to a function which is 
true for all values. 

Conversely, each set X gives rise to a propositional function : t is an ele- 
ment of X. This function is true precisely when t belongs to X. A Boolean 
algebra of sets thus leads to a Boolean algebra of propositional functions. 

Because of the parallel between propositional functions and sets, one can 
employ geometric set diagrams, as in Figs. 1 and 2, to reason about proposi- 
tional functions. In logic they are called "Venn diagrams." 

Quantifiers. The operation of forming the intersection of many sets 
has an analogue for sentences or propositional functions. As for sets 

n 

(Chap. 1, Sect. 1) f"| x t denotes X\ D x 2 (1 • • • D x n . When the x's are 

sentences, this is the new sentence: "every one of the z's" or "for every t, 
x t ." The range of t may be over an infinite set. When the range is under- 
stood, one writes simply f] x t . Similarly, if f(t) is a propositional func- 

t 
tion and t ranges over all values for which f(t) has meaning, then Q f(t) is 

t 

read: "for every t, f(t)." Alternative notations for C\ f(t) are V/(0 and 

t t 

Ylf(f)- One terms P) a quantifier. There is an analogous interpretation 

t t 

of (J x t and [J f(t) ; the first is read "for some t, x t " and the second "there 

t t 

exists a t such that /(/)•" An alternative notation for |J is 3 ; [J is also 
called a quantifier. l l t 

Implication. The statement "x implies y" is capable of various inter- 
pretations, of which three will be discussed here : material implication, con- 
ditional implication, and strict implication. Throughout, x, y, - • • denote 
sentences forming a Boolean algebra B. 

Material implication. From the sentences x, y one forms the new sen- 
tence: (( x implies y" as the sentence x' U y. This is called material impli- 



BOOLEAN ALGEBRA 11-07 

cation. One often writes x Z) y or x =» y f or this implication : 

(24) x Z) y = x' U ij = x => y. 

If x and y are propositional functions x(t), y(t), then they can be represented 
as sets X, Y. The sentence is then a propositional function which is true 
for alH if X' U Y = I; that is, if X C Y. The notation x Z) y is therefore 
unfortunate. Material implication is the basis for most mathematical argu- 
ments, but it is criticized as permitting such statements as "if Iceland is an 
island, then fish can swim" to be judged true. 

Conditional implication. For each pair of sentences x, y a new sentence 
y/x is formed and is read "if x then y" or "y if x." It will be assumed that 
x 9^ 0. This is called conditional implication. The significance of the new 
sentence is indicated by certain postulates : 

(25) x/x = 1, 

(26) y/x = implies y C\ x = 0, 

(27) (y C\z)/x= (y/x) fl (z/x), 

(28) z/{x fl y) = (z/x)/(y/x), 

(29) (l@y)/x=l®(y/x), 

(30) for every £, y there is a z such that z/x = y, if x t^ 0. 

Conditional implication is designed to fit the needs of the theory of probability. 
When x is false, it may happen that y/x is true or that y/x is neither true 
nor false. 

One can verify that postulates (25) to (28) are satisfied by material im- 
plication, but that (29), (30) are not. However, (29) is a reasonable de- 
mand to make on an implication, and it is valuable in theory of proba- 
bility. Postulate (30) requires that B contain sufficiently many sentences 
so that one can always solve the equation z/x = y for the sentence z. A 
Boolean algebra which has an operation x/y satisfying postulates (25) to 
(30) is called an implicative Boolean algebra. It can be shown that an im- 
plicative Boolean algebra cannot be atomic (Sect. 4) but that one can 
always construct an implicative Boolean algebra containing any given 
Boolean algebra. 

Strict implication is defined as follows. The strict implication x implies 

y holds if and only if the material implication is a tautology (i.e., x Z) y 

= 1) and this is true if and only if y/x = 1. When x and y are interpreted 

as sets then the equation x Z) y = 1 can be interpreted as stating that x 

is contained in y. This relation has the following alternative notations: 



11-08 GENERAL MATHEMATICS 

x ^ y, y ^ x, x Q y, y 3 x, x d y, y Z> x. The last two notations are un- 
fortunate since they almost reverse the interpretation of the implication 
symbol. 

4. CANONICAL FORM OF BOOLEAN FUNCTIONS 

Let a Boolean algebra B be given, with operations U, D, and ' as in 
the first definition of Sect. 1. By a Boolean function or Boolean 'polynomial 
in n variables x\, • • •, x n is meant an expression constructed from the n 
variable elements x 1} • • • , x n by the three operations U , C\ , ' . For example, 

MP (x' U z') 

is a Boolean polynomial in three variables. It would appear at first that 
such expressions can be made arbitrarily long and hence that, for fixed n, 
there are infinitely many polynomials. However, by the rules of the alge- 
bra, each polynomial can be simplified, and there are precisely 2 2 poly- 
nomials for each n. For example, there are four polynomials in one variable 
x: x, x', = x f~l x', 1 = x U x' . 

If two Boolean polynomials in Ji, • • •, x n are given, one may wish to 
determine whether they are the same ; that is, whether one can be reduced 
to the other by applying the algebraic rules. In order to decide this, one 
reduces both polynomials to a canonical form, as described below. If both 
have the same canonical form, they are the same; otherwise, they are un- 
equal polynomials. 

Definitions of Canonical Form and Minimal Polynomials. By a 
minimal polynomial in x\, • • • , x n is meant an intersection of n letters in 
which the *th letter is either X{ or x'{. 

Examples. There are four minimal polynomials in x, y: 

x D y, x' D y, x fl y' , x' D y'. 

Similarly, there are eight minimal polynomials in x,y,z: 

x n y n z, x r\ y' n z, x' n y n z, x' n y' n z 
x n y n z', x n y ' n z', x' n y n z', x f n y f n z\ 

There are 2 n such minimal polynomials mx\, • • • , x n . 

By a polynomial in canonical form is meant a polynomial which is either 
or else is a union of distinct minimal polynomials. (The order of the 
terms can be specified, but this is of no importance since U is commuta- 
tive.) For example, 

(x fl y) U {x' D y'), MpM y') U (x f fl y') 
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are in canonical form. Every polynomial can be written in a unique canoni- 
cal form, so that equality of two polynomials holds if and only if they have 
the same canonical form (Ref. 3). 

Reduction to Canonical Form. A given polynomial can be reduced 
to canonical form by the following steps : 

(i) Moving all primes inside parentheses by (12) ; 

(ii) Moving all caps ( D 's) to the inside of parentheses by the first rule 
(4); 

(iii) Simplification of terms by rules (1), (2), (9), (10), (11), (13), so that 
one finally obtains a union of terms, each of which is a minimal polynomial 
in some of the x's; 

(iv) Adjoining missing x's to the minimal polynomials by inserting 
x U x' = 1 for each such x; 

(v) Applying steps (ii) and (iii) again. 

Example. 

M(yU z)] U [(x U y) (y' U z)'] 

= [(x f)y)U (xCl z)] U [(x U y) II (y D z')] 

= (x (1 y) U (a: D z) U (x D y (1 z') U (y H y C\ z') 

=\(x n y) n (*' u z)\ u [(x n z) n w u y )] u {x n y n z') 

U [(y n z') (1 {x' U re)] 

= (a; n i/ n 2') u (x n i/ n s) u (x n y' n 2) u (x n i/ n z) 
u (cr n ?/ n z') u (^' n ?/ n z r ) u (x n ?/ n z') 

= (x n ?/ n 2') u (x n ?/ n 2) u (x n ?/' n 2) u (x' n i/ n 2 ') 

5. STONE REPRESENTATION 

Let a Boolean algebra 5 be given. Then it is possible to find a set S 
and to define a one-to-one correspondence between the elements x, y, • • • 
of B and the certain subsets X, Y, • • • of S in such a fashion that if x 
corresponds to X and y to Y, then x U y corresponds to X U Y, x D y to 
X fl 7, a/ to X', to the empty subset 0, and 1 to S itself. Thus every 
Boolean algebra can be represented as (is isomorphic to) the Boolean alge- 
bra of certain subsets of a set $. This is the Stone representation. 

If B has only a finite number m of elements, then B can always be repre- 
sented as the Boolean algebra of all subsets of a given set S. Furthermore, 
m must be of the form 2 W , where n is the number of elements in S. If B\ 
and B 2 are Boolean algebras both having m elements, where m is finite, 
then Bi and B 2 are isomorphic. 
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Stone Representation Theorem. An infinite Boolean algebra B can 
be represented as the Boolean algebra of all subsets of a set S if and only if B 
is atomic, complete, and distributive. These properties are denned as 
follows: 

An element a of B is called an atom if the intersection x f) a of a with an 
arbitrary element x of B is either a or 0. If, for each x other than in B, 
there is an atom a such that x D a = a, then B is said to be atomic. In the 
representation of B as a; class of sets, the atoms correspond to sets each 
containing one point. 

A Boolean algebra B is said to be complete if every subset A of B has a 
least upper bound (Chap. 1, Sect. 7). The least upper bound is then 
unique; it can be denoted by 

U(A) or (J a, 

and is also called the union of A. 

A Boolean algebra B is said to be distributive if, whenever \J (A) exists, 

(3i) ^nU(i) = U(^n«) 

a£ A 

for every /3 in B. 

6. SHEFFER STROKE OPERATION 
In a Boolean algebra B let 

(32) x\y = x' U y'. 

If x and y are sentences, x\y is the sentence "either not x or not y." One 
can then prove that 

(33) x\x = x' = l@x, 

(34) (x\y)\(x\y) = x D y, 

(35) (x\x)\(y\y) = x U y, 

(36) x\(y\y) = x' U y, 

(37) z|Gc|aO = 1. 

Accordingly, all the operations of the Boolean algebra can be expressed in 
terms of the Sheffer stroke operation. This proves to be of value in the design 
of electronic digital computing machines, which compute in the scale of 
two (see Ref . 6) . 
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1. FUNDAMENTAL CONCEPTS AND RELATED PROBABILITIES 

Postulates. The probability that an event will occur is a real number 
between and 1. If re denotes the sentence, the event will occur, then 
Pr(x) denotes the probability that the event will occur. Thus Pr(x) is the 
probability associated with the sentence x. Consider a Boolean algebra B 
of sentences (see Chap. 11) in which is interpreted as the sentence as- 
sociated with an impossible event and 1 is interpreted as the sentence 
associated with a certain event. This treatment will (a) show how some 
probabilities can be computed from others; (b) study the relations between 
probabilities of sentences connected by the words and, or, not, if (denoted 
respectively by D , U , ', /) . 

12-01 
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Assume that the following postulates hold: 

(1) Pr{x) is a non-negative real number if x is in B. 

(2) If x\, x 2 , • • • , are in B and xi D Xj = when i ^ j where i, j = 1, 2, 



then \J Xk is in B and 



k=\ 



(00 \ 00 

U **) = £ p r{x k ). 
k=\ / k=\ 



(3) Pr{l) = 1. 

(4) Pr(x n y) = Pr(x) Pr(y/x). 

If Xi fl Xj = 0, i.e., if X{, Xj cannot both occur, then X{, Xj are said to be 
mutually exclusive and the events associated with them are also said to 
be mutually exclusive. Thus postulate (2) states that the probability 
that at least one of a set mutually exclusive events will occur is the sum 
of their probabilities. The following theorems are consequences of the 
above postulates. 

Theorem 1. ^ Pr{x) ^ 1. 

Theorem 2. Pr(0) = 0. 



(n \ n 

U x k ) = £ Pr( x k) if X{ D Xj = whenever 
A;=l / fc=l 



i ?* j. 



Theorem 4. Pr(x l) y) = Pr(x) + Pr(y) — Pr(x D y). 

Theorem 5. Pr(x') = 1 — Pr(x). 

Theorem 6. Pr(x D y') = Pr(x) — Pr{x D y). 

Theorem 7. If X\, x 2 , • • •, x n are mutually exclusive (i.e., X{ fl x 3 - = 
w/ien i 9^ j) and exhaustive (i.e., #1 U rr- 2 U • • • Ux n = 1) and equally 
likely (i.e., all Pr(xk) are equal) i/ien Pr{xk) = 1/n. for & = 1, 2, • • •, n. 

Example 1. As an illustration of Theorem 7 consider a coin which is 
about to be tossed, and let x\ be the sentence "the coin will turn up heads" 
and x 2 be the sentence "the coin will turn up tails." If the coin is not 
loaded, one says that it is honest and assumes that the hypotheses of 
Theorem 7 hold. Then Pr{x x ) = Pr(x 2 ) = Yi- 

Example 2. Next consider an honest die which is about to be thrown 
and let Xk be the sentence "the face numbered k will turn up" where 
k = 1, 2, • • •, 6. Again one assumes that the hypotheses of Theorem 7 
hold and concludes that Pr(xk) = }/% for k = 1, 2, • • •, 6. The probability 
that the die will turn up an odd number is given by Theorem 3. Thus 

Pr{x x Ux 3 U x 5 ) = P(h) 4- Pr(x 3 ) + Pr(x 5 ) = | = •§-. 

Example 3. Next let x = X\ U x 3 U x 5 , y = Xi U x 2 U x s and note 
that Pr(x) is the probability that the die will turn up an odd number and 
Pr{y) is the probability that it will turn up a number less than 4. It will 
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be instructive for the reader to check that 

x' = x 2 U X4 U x G , y' = £4 U £5 U Xq 

x D y = x\ U x 3 , x U y = xi \J x 2 U x% I) x& 

and also to check Theorems 4, 5, and 6 for this x and y. To compute the 
conditional probability Pr(y/x), i.e., the probability that the die will turn 
a number less than 4 if it turns up an odd number, use postulate (4). Thus 

Pr{x D y) = -g- = Pr{x)Pr{y/x) = ^Pr(y/x), 

and hence 

Pr(y/x) = -§. 

Example 4. Next consider three boxes and let Xk denote the sentence 
"the kth box will be selected" where k = 1, 2, 3. If one of the boxes is 
selected at random, this is interpreted to mean that the hypotheses of 
Theorem 7 hold and hence that 

PrM = Pr(x 2 ) = Pr(x 3 ) = £. 

Suppose further that the first box contains two silver coins, the second 
contains one silver coin and one gold coin, the third contains two gold 
coins, and that a coin is drawn at random from the box which has been 
selected. Let y denote the sentence "a gold coin will be drawn from the 
box which has been selected." Then 

PrteM) = 0, Pr(y/x 2 ) = -J-, Pr(y/x 3 ) = 1. 

Now suppose that this experiment has been performed and that the coin 
has been examined and found to be gold. On the basis of this information 
what is the probability that the coin came from the third box containing 
the two gold coins? One interprets the answer to this question as the 
conditional probability Pr(x 3 /y), i.e., the probability that the third box 
was drawn if the coin was observed to be gold. It will be instructive 
for the reader to verify that Pr(x 3 /y) = % with the aid of the following 
theorem which is called Bayes's theorem and which is a consequence of 
the above postulates. 

Theorem 8. Bayes's Theorem. Ijx x , x 2 , • • ■ , x n are mutually exclusive, 
exhaustive and distinct from 0, then for any y one has 

n 

Pr(x{/y) = Pr(xi)Pr(y/xi)/ ]£ Pr(x k )Pr(y/x k ) 

k=i 

if the denominator is not 0. 
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Independence. The sentences X\, x 2 , • - • ,x n are said to be independent 
if 

Pr( fl Xk) = Pr(x x ) • • • Pr{Xn) 
\k=i / 

and if a similar equation holds for every subset of x i} x 2 , • • -, x n . 
Thus when n = 3 one has 

Pr(xi D x 2 fl £3) = Pr(x 1 )Pr(x 2 )Pr(x 3 ), 

Pr(xi D # 2 ) = Pr(^i)Pr(^ 2 ), 

Pr(a; 2 D £3) = Pr(x 2 )Pr(x 3 ), 

Pr^ D £ 3 ) = Pr(xi)Pr(x 3 ). 

If #i, £ 2 are independent and Pr{x{) ^ 0, Pr(x 2 ) 5* then 

Pr(#i D a; 2 ) = Pr(x\)Pr{x 2 ) 

= Pr(xi)Pr(x 2 /x 1 ) 

= Pr(x 2 )Pr(xi/x 2 ), 
and hence 

Pr(x 2 /xi) = Pr(x 2 ) and Pr(xi/x 2 ) = Pr(x{). 

2. RANDOM VARIABLES AND DISTRIBUTION FUNCTIONS 

Consider a physical experiment which is designed to result in a real 
number. This number is subject to certain random fluctuations since in 
all physical experiments one expects experimental errors to be present. 
The result of the experiment is interpreted as a random variable X. For 
a mathematical definition of a random variable, see below. Let x\ (for 
any real number X) denote the sentence the experiment will produce a num- 
ber less than X, i.e., the sentence X is less than X. Then the probability 
Pr(x\) is a function F of the real variable X called the distribution function 
ofX. Thus PrOc x ) = P(X). 

If Xi < X 2 then 

:Pr(x X2 fl x' Xl ) = Pr(x X2 ) - Pr(x Xl D x Xi ) = F(K 2 ) - P(X X ) 

is the probability that X is greater than or equal to Xi, but less than X 2 . 
Thus when F is known, one can find the probability that X lies in a given 
interval. 

In Chap. 11 it was noted that the elements of a Boolean algebra can be 
interpreted as sets of points of some space. Thus one interprets x Xl (1 x\ 2 
as a set and Pr{x\ 2 D x'xj as the probability of obtaining a point of this 
set, that is, the probability that the experiment will select a point £ of 
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this set. Imagine that the number which the experiment produces is 
determined by the point £ selected and hence that X is a function of £. 
Then x\ is the set of all points £ for which X(£) < X. The only restrictions 
placed on the function X are that it is real valued and that each of the sets 
x\ shall belong to B. Such a function is said to be measurable with respect 
to B. The measure of a set x\ is defined as the probability Pr(x\). A 
random variable X is a function which is measurable with respect to B. 

Let X be a random variable, let x\ be the set of points £ for which X(£) 
^ X, and denote Pr(x\) by F(\+). Then it can be proved (using postulate 
2) that x\ is in B for all real X. Moreover F(\-\-) is the limit of F(n) as 
H approaches X through values greater than X. If /x approaches X through 
values less than X then the limit of F(n) is F(\). Furthermore, F is a 
nondecreasing function for which 



lim F{\) = F(-oo) = 0, 



lim F(\) = F(+x>) = 1. 



The above properties characterize the distribution function of a random 
variable. 

Example 1. As an illustration of a random variable let x be any element 
of B and let 

f 1 if £ is in the set x 

MS) = 

1 if £ is not in the set x. 

Then \}/ x is called the characteristic function of the set x and is interpreted 
as the random variable which takes on the value 1 when x succeeds and the 



mF(X) 



Pr(x') 



Fig. 1. Distribution function for Example 1. 

value when x fails. The distribution function F of the random variable 
\p x is the following (see Fig. 1) : 



F(k) 



fO ifX^O 

Pr{x') if < X ^ 1 
11 if 1 < X. 
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Example 2. Consider a die and let Xk denote the sentence the face num- 
bered k will turn up. Let 



X = f Xl + 2+ X2 + 3^ 3 + 4^ 4 + 5^ 6 + 6£ 



*6' 



If the face numbered k does turn up then this will assign the value 1 to 
\[/ Xk and the value to the remaining characteristic functions and hence X 



A*XX) 




Fig. 2. Distribution function for Example 2, random tossing of a die. 

will take on the value k. Thus X is the random variable which takes on 
the value which the die turns up (see Fig. 2) . 

It can be proved that sums, products, and differences of random variables 
are again random variables. Furthermore any real number is a random 
variable. 

Example 3. The number \/2 is the random variable whose distribution 
function F is given by (see Fig. 3) : 



^(X) = 



IF(\) 



if X < \/2 

1 if \/2 ^ X. 




Fig. 3. Distribution function for Example 3. 



3. EXPECTED VALUE 

If X is a random variable associated with some experiment and if the 
experiment is repeated a large number of times, then one should expect the 
average of the numbers obtained to be very close to some fixed number 
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E(X) which is called the expected value of X. In order to make this idea 
more precise the following definition is introduced. The random variables 
Xi, X 2 , • • •, X n are said to be independent provided £i,\ u x 2< \ 2 , • • •, x n> \ n 
are independent for all Xi, X 2 , • • •, X n where Xk,\ k is the set of points for 
which X fc (£) < X fc . 

As an illustration of independent random variables, consider a pair of 
honest dice. Let X x denote the random variable which takes on the value 
resulting from the throw of the first die and X 2 denote the random variable 
corresponding to the second die. It is reasonable to assume that X\ and 
X 2 are independent. Thus we assume that the occurrence of a number less 
than Xi = 3 on the first die and the occurrence of a number less than X 2 
= 5 on the second die are independent events; similarly, for other choices 
of Xi and X 2 . Next let X 3 = Xx + X 2 . Then X x and X 3 are dependent 
random variables. 

Weak Law of Large Numbers. Now let X be an arbitrary random 
variable and let X\, X 2 , • • • , X n be independent random variables all having 
the same distribution function as X. Let x t<n be the set of points £ for 
which 

I (Xx(£) + X 2 (£) + • • • + X„(0)/n - E(X) | ,< e. 

Then x t>n is interpreted as the sentence the average of Xx, X 2 , • • •, X n will 
differ from E{X) by less than e. One might expect that 

lim Pr(x e , n ) = 1 for every e > 

n — > oo 

and that there is only one choice of E(X) for which this limiting probability 
is 1. This, as a matter of fact, is the case and this result is called the weak 
law of large numbers. Roughly the weak law of large numbers states that if 
an experiment is repeated a large number of times then it is very likely that the 
average of the results will differ only slightly from the expected value. The 
expected value E{X) exists for a large class of random variables but not for 
all random variables. 

Properties of Expected Value. 

Theorem 9. E(\X + nY) = \E(X) + pE(Y) if X, n are real numbers 
and X, Y are random variables for which E(X), E(Y) exist. 

Theorem 10. If E(X), E{Y) exist and X (£) ^ F(£) for all £ then E(X) 
^ E(Y). 

Theorem 11. If $ x is the characteristic function of the set x then E(\f/ X ) = 
Pr(x). 

With the aid of Theorems 9 and 11 one can compute expected value for 
certain random variables called simple random variables. 
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A random variable X is simple if it has the form 

n 
X — E Watt 

where each X k is a real number and each \p Xk is the characteristic function of 
the set x k . 
Theorem 12. 

E(X) =e(J2 \ k * xk ) = E \ k Pr(x k ). 

Theorem 10 is used to approximate expected value for a much larger 
class of random variables called bounded random variables. The real num- 
bers X, n are called bounds for a random variable X if X ^ X(£) ^ ju for 
all £. When the bounds exist X is said to be bounded. 

Theorem 13. If X, n are bounds for X and if X = Xo < Xi < • • • < X n 
= ju then E{X) exists and 

n n 

E X*_i(F(Xjb) - fto-i)) ^ E(X) tz E X fc (^(X fc ) - ^(Xt-i)) 

w/iere F t's £/ie distribution function of X. If each \ k — Xfc_i < e, 2/iew #je 
extreme members of the inequalities differ by at most e. 
Theorem 13 is readily established as follows. Let 



then 



and 



4>k = +* Xk n x\ k _ x 

n n 

E fa — i, E %fa = x, 
fc=i fc=i 

\ k -lfa ^ X<£fc < ^kfa 

tf fob) = Pr(xx 4 D x\ k J = F(X*) - F(\ k -i) 



and hence the inequalities follow from Theorems 9 to 11. The difference 
between the extreme members of the inequalities is : 

n n 

E (x* - \k-iW(\ k ) - f(x fc _i)) < E <nw - F{\ k _ x )) 
£=1 k=i 

= €(^0*) - FQi)) = 6. 
If i* 7 has a continuous derivative/ (i.e., dF(\)/d\ = /(X)) then 
F(X fc ) - F(X*_i) = /W(X fc - X fc -i) 
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where \k-i < Mfc < X& and 



lim 2 \k(F(k k ) ~ F(X fc _0) = lim 2 X*/(Mfc)(Xfc - X*_i) 

e -* fc=l « -» &=! 



= J W(w) 



Thus 

Theorem 14. // the distribution function F of a random variable X has 
a continuous derivative f and X, n are bounds of X then E(X) exists and 



#(X) = I uf{u) du = I udF{u). 



Theorem 15. // the distribution function F of a random variable X has a 
derivative f then E(X) exists and 



E{X) = I uf(u) du = I w dF{u) 

J —00 ^ —00 



whenever the integral exists. 

Stieltjes and Lebesgue Integrals. The two cases which arise most 
frequently in practice are the simple random variables and the random 
variables whose distribution functions have continuous derivatives. In 
the first case the expected value is computed by means of Theorem 12 
and in the second case by Theorem 15. The integral on the right of the 
equation of Theorem 15 can be assigned a meaning even when / does not 
exist. In the case of a bounded variable this integral is defined to be the 
limit of the approximations given in Theorem 13. A meaning can also 
be assigned in certain unbounded cases. This integral is called a Stieltjes 
integral. Another integral expression for E{X) is 



E(X) =j 



XdPr. 



This is called a Lebesgue integral and it is also defined in terms of the 
approximations of Theorem 13. 

The terms expectation and mean are often used as synonyms for expected 
value. 

Probability Density and Joint Distribution. The derivative / of 
F is called the probability density. When the density is given the distribu- 
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tion function can be computed by the formula 



F(\) = f f(u) du. 

J nn 



See Figs. 4 and 5, Sect. 5. 

The joint distribution of two random variables X\, X 2 is a function F 
such that F(\i, X 2 ) is the probability that Xi < Xi, andX 2 < X 2 . If the 
joint distribution has a density / then 



F(Xi, X 2 ) = I I /(mi, m 2 ) dux du 2 . 

J — rr, J — rr\ 



— 00 " — 00 



if Fi, F 2 are the distribution functions of X\, X 2 , and /i, / 2 are the cor- 
responding densities then 

^l(Xl) = I I. /(Mi, M 2 ) C?M! C?W 2 = I /i(Mi) C?Mi, 

»^— 00 «' — oo ■' — 00 

F 2 (h 2 ) = I I /(mi, m 2 ) du 2 dui = I / 2 (m 2 ) du 2 , 

J —oo J —oo J 

/i(mi) = I /(mi, m 2 ) du 2 , f 2(112) = I /(Mi, M 2 ) du x . 

•^—00 •' — 00 

The expected value of the product X\X 2 is 

/oo _00 
I MiM 2 /(Mi, M 2 ) dMi dM 2 . 
—00 •' —00 

If Xi, X 2 are independent then 

^(Xi, X 2 ) ->(Xi)F(X a ) 
and 

/(Ml, M 2 ) = /(Mi)/(M 2 ). 

Furthermore : 

Theorem 16. 7/ Xi, X 2 are independent then E{XxX 2 ) = E(X 1 )E(X 2 ). 

Two random variables X u X 2 for which E{X X X 2 ) = E(X 1 )E(X 2 ) are 
said to be uncorrelated. Thus Theorem 16 states that independent random 
variables are uncorrelated. This result holds even when there is no joint 
probability density. The converse is not true. That is, random variables 
may be uncorrelated, but not independent. 
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As an illustration of a pair of random variables which are dependent but 
uncorrelated, consider an honest die whose faces are numbered respectively 
— 3, —2, — 1, 1, 2, 3. Let X x denote the random variable which takes on 
the value resulting from the throw of this die and let X 2 = Xi 2 . Then 

E{X X ) = 0, E(X 2 ) = - 1 /, 

E(X l X 2 ) = E(X 1 3 ) = = E(X X )E{X 2 ) 

Hence Xi and X 2 are uncorrelated but they are clearly dependent. 

4. VARIANCE 

Theorem 17. If f is a function of a real variable X with at most a finite 
number of discontinuities and if X is a random variable with distribution 
function F, then f(X) is a random variable and 



W(X)) = f f(u) dF(u) 

J — QO 



whenever the integral exists. 

A special case of this formula is the following: 

If 

E(X) = ix 
and 

E((X - m) 2 ) = f (u - m) 2 dF(u) = E(X 2 ) - E 2 (X) = <r 2 (X), 

then <r 2 (X) is called the variance of X and the positive square root of the 
variance, <r{X), is called the standard deviation of X. 
The Properties of Variance. 

Theorem 18. a 2 (X + X) = a 2 (X), <r 2 (XX) = XV(X). 

Theorem 19. If X\, X 2 , - • • , X n are independent random variables, then 

^(X, + X 2 + • • • + X n ) = a 2 {X x ) + a 2 (X 2 ) + • • • + a 2 {X n ), 
a 2 ( Xi + X 2 +---+X n \ = l (r 2 (XQ + (r 2 (X 2 )+v--+cr 2 (X w ) 
V n / n n 

If x ( is the set of all points £ such that |X(£) — n\ < e where n — E(X) 
then x' e is the set of all £ such that |X(£) — /x | ^ e. Moreover the in- 
equality 

eV«' f tt) ^ (*(*) - M) 2 

can readily be verified when £ is in x t and when £ is in x' f and hence this 
inequality holds for all £. Thus by Theorems 9 to 1 1 it follows that 

E(<fy x ,) = e 2 Pr(x' e ) ^ E((X - M ) 2 ) = <r 2 (X), 
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and therefore 

Pr(x t ) ^ 1 - (r 2 (Z)/e 2 . 

This inequality is called Tchebysheff's inequality. By combining Tcheby- 
sheff's inequality with Theorem 19, one obtains: 

Theorem 20. If X\, X 2 , • • •, X n are independent random variables with 
common mean n and common variance <r 2 and if x f<n is the set of points £ 
for which 

| (Xjfc) + X&) + • • ■ + X„($))/n - m| < e 
then 

Pr(x e , n ) ^ 1 - a 2 In? 
and 

lim Pr(x e , n ) = 1 for every e > 0. 

n — * =o 

The Strong Law of Large Numbers. The first part of Theorem 20 
gives a crude approximation for the probability that the average will 
differ from the common mean by less than e. Recall that the second part 
of this theorem is the weak law of large numbers. The reasoning by which 
one arrived at this result is of course circular but this circularity can be 
avoided. The strong law of large numbers is the following: 

Theorem 21. If X\, X 2 , • • • , X n are independent random variables with 
common expected value n and common variance a 2 and if x is the set of points 
£ for which 

Vm xi(a + x 2 (a+--.+ x»(o ____ 

n— »oo yi 

Then Pr(x) = 1. 

Even though Pr(x) = 1 it is not in general true that x = 1. If an 
element x of B is such that Prix) = 1, then x is said to be almost certain. 
The strong law of large numbers states that it is almost certain that the 
limit of the average is the common expected value. 

The following example will help one understand the distinction between 
certain and almost certain. Let X be a random variable with distribution 
function F defined as follows : 



F{\) = 



if X ^ 

X if ^ X ^ 1 

1 if 1 < X. 



Then it is almost certain, but not entirely certain, that X will take on a 
value distinct from Y%. 
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5. CENTRAL LIMIT THEOREM 



Distribution of Sums and Averages of Independent Random 
Variables. Consider 

e ixt = cos Xt + i sin Xt 

where i 2 = — 1 and t is a parameter. This exponential converts the real 
random variable X into a complex valued random variable. The expected 
value of the latter random variable is denned in a natural way to be 

E(e iXi ) = E(cosXt) + iE(smXt) = </> x (0- 

The advantage of the exponential is that it converts a sum into a product 
and hence enables one to make use of the condition of independence. Thus 
if X, Y are independent then it can be shown that e lXt , e lYt are independent 
and hence by Theorem 16 

'*x+y(0 = E{e iXt e iYt ) = 4>x{t)4> Y (t). 

The advantage of the factor i is that it produces a bounded random variable 
and insures the existence of the expected value for all real values of t. 
The advantage of the parameter t is that it produces a function in terms 
of which one can compute the distribution function. Thus <(>x is a function 
of the parameter t called the characteristic function of the random variable X. 
Unfortunately the phrase, characteristic function, has two distinct mean- 
ings in the theory of probability, namely, characteristic function of a set of 
points and characteristic function of a random variable. 

Computation of the Characteristic Function of a Simple Random 
Variable. Let 

n 

X = X] ^k^Xk, 

fc=l 

where x\, x 2 , • • •, x n are mutually exclusive and exhaustive. Then 

e ixm = £ e a»^ (€) 

for all £, since if £ lies in Xk, then yp Xk (Q = 1 and the remaining characteristic 
functions have the value and hence both sides of the equation become 
e iXkt . From this it follows that 

n 

E(e iXt ) = 2 Pr(x k )e*** = <f>x(t). 

k=l 

As a special case consider the simple random variable yp x . One can write 

$x = 0-tfv + 1-ifcc 
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and hence 

.**,(*) =q.+ pe u 

where p = Pr(x), q = Pr{x'). 

Next compute the characteristic function for the sum xp Xl +^2 + • • • 
+ ~^x n where X\, x 2 , • • • , x n are independent and Pr(xk) = p, Pr(x'k) = q 
for each k. Then 

n 

***,+*,,+ -+M0 = II 0***(O = (?+ Pe")". 
If X has the distribution function jP then the characteristic function is 



4>x(t) = E(e iXt ) = J e A ' <JF(X). 



This transforms the function i* 7 into the function <j>x (essentially the 
Laplace-Fourier transform). The inverse transform is 



\{F{\) + FQi+)) = -^ lim f 

2irl h -* » J _h 



h e il - <f>x(t)e- xt 
dt. 



To see why this is the case note that 

I n h e il _ e int e —i\t 2 ~cc e it e int e —i\t 

— : lim I dt = — I dt 

2irih -> * J_ h t 27rw_ 00 t 

1 if ft < X, 
J if ju = X. 
if X < fx. 

This formula is verified by converting the integral into integrals of the form 

sin mt 



1 



t 



dt 



by means of the relation e imt = cos mt + i sin mt. Now compute the in- 
verse transform for a simple random variable 

n 

X = 2 ^xic, 
fc=l 

where X\, x 2 , •• •, x n are mutually exclusive and exhaustive. Since 

n 

2 JMa*) = 1, 

/b=i 
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then 

n n 

Y,Pr(xi)e U ~ T, Pr(x k )e iX ^e- iXt 
ire— <t>x{t)e i r k =i fc=i 

— I dt — — ; I dt 

27rtJ_ QO t 27rw_ 00 



t 
Pr(x k ) r x e lt - e tX *V x * 



< 



dt 



fc = l 27TZ «/_ a 

= £ Pr(x k ), 
x*< x 

if X 5^ any X/t. The final sum is the probability that X will take on a value 
less than X and hence this sum is equal to 

F(\) = F(X+) = i(F(X) + F(X+)). 

If X equals some X&, then the corresponding term Pr(xj c )/2 must be added 
and again the result is %(F(X) + F(X+))- The proof for an arbitrary 
random variable X consists in approximating X by a simple random vari- 
able. In the general case the integral from -« to <» may not exist, and 
one has to resort to integrating from —h to h and then passing to the limit. 
Binomial and Poisson Distributions. If F n is the distribution func- 
tion of 

X = f Xl + fa* H h ^x„ 

where Xi, x 2 , • • • , x n are independent and Pr(x k ) = p, Pr{x' k ) = q for each 
k then F n (\) is the probability that less than X of the events x 1} • • •, x n 
will succeed. If X ^ any k then 

1 z* 00 e u - 4>x{t)e~ iXt 
2iriJ_ o0 t 

(q + p) n e u -(q + pe u ) n e~ iU 



--■) 



t 



dt 



" /n\ , . 1 r°° « ~ e lkt e~ lU 

£ q k V n I dt 

c=0 \k/ 2iriJ_ o0 t 



k= 



= e ( !>y-*, 

&<x V/c/ 



where (£) is the number of combinations of n things taken k at a time. 
If X = some k, then the corresponding term ( k l )p k q n ~ k /2 must be added. 
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This is called the binomial distribution. To obtain an approximation to 
this distribution for small p and large n set p = n/n and let n become 
infinite. The limiting distribution F is given by 

This is called the Poisson distribution. 

Normal or Gaussian Distribution. Next consider a sum U n = 
Xi + X 2 + • — h X n of independent random variables with a common 
distribution , function F. Denote the common expected value by n and 
common variance by a 2 . If Y k = (Xk — m)A and V n = (U n — nn)/a-\/n 
then (see Theorems 18 and 19), 

Yn = (F 1 H-7 2 +..-+F n )/V^, 

E(Y k ) = = tf(7«), cr 2 (7*) = 1 = a\V n ). 

Moreover 

n 

*f„(0 = II # ex P (iY k t/Vn ) = # n exp (iY x t/Vn ) 

= # re (l + *T„*/Vn - F w 2 ^ 2 /2w +•••), 
= (1 + -i 2 /2n +•••)", 

= ((1 -t 2 /2n+...)- 2n ' t2 )- t212 . 
Hence 

lim Fn (O = e- t2 ' 2 

n —* « 

and the limiting distribution is 

1 r +« e « _ e -« s /2 e -iX« 

$(X) = tft. 

2« ./__«, t 

To obtain a simpler form for $ compute its derivative. Thus 



d$ i r 

— (X) = — 

rfX 27T J_c. 



,-t 2 /2 e -i\t 



Since f 2 + 2t"X£ = (< + *X) 2 + X 2 , set u = £ -f- tX, cfai = d£ and obtain 

<& i r +0 ° „ , ln e~ x2/2 r +co „„ Ae" x/2 



_ (X) = — I e -<'+*» ' 2 e -* / 2 dt = I e" M2/2 rf W = 

dX 27T J_ w 27T •/_«, 



2tt 
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where 



Hence 



-L 



~ u2 > 2 du. 



$(X) = — fe-* 12 du = — f e- uZ ' 2 



du + C. 



Since $ (— °°) = this means that (7 = 0. Moreover 

A r x A 2 

2tJ_ 00 2tt 

Therefore A = V2r and 

This limiting distribution is called the normal distribution or gaussian 
distribution and is shown in Fig. 4. 



A $(X) 




Fig. 4. Normal distribution function. 



Theorem 22. Central Limit Theorem. IfXi, X 2 , • • • are independent 
random variables with a common distribution function F common expected 
value fx and common variance a- 2 then the distribution function of (Xi + X 2 
+ • • • + X n — nn)/a\/n converges to the normal distribution function as n 
becomes infinite. 

If s = Xa/-\/n, i.e., X = ss/n/a- then the probability that (Xi + X 2 
-\ — • + X n — nn)/cr-\/n < X is equal to the probability that (Xi + X 2 
H h X n )/n — n < s and approximately equal to 

$(X) = 3>(sVn/<r). 
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Figure 5 shows the probability density corresponding to the normal 
distribution function. 




Fig. 5. Probability density for the normal distribution function. 



6. RANDOM PROCESSES 

A continuous random process is a function X which assigns to every real 
number t, a random variable X t . If i ranges only over the integers then 
the process X is said to be discrete and if t ranges only over the positive 
integers then the process X is simply a sequence of random variables. 
Consider complex valued random variables, i.e., random variables of the 
form X = Xi + iX 2 where X\, X 2 are real. The complex conjugate of 
X is Xi — iX 2 and is denoted by X. The inner product of two random 
variables X, Y is denoted by (X, Y) and defined by the equation 

(X, Y) = E(XY). 

The covariance function R of a process X is defined by the equation 

R(t, r) = (Xt+r, X t ). 

If R depends only on t and not on t, then the process is said to be stationary 
in the wide sense. A physical example of such a process is the phenomenon 
of noise. In the mathematical model (i.e., the process X) the variable t 
is interpreted as time. The process can be envisioned as being composed 
of simple harmonic oscillations in which the amplitudes associated with the 
various frequencies are selected in accordance with a certain random pro- 
cedure. A simple harmonic oscillation of frequency X is represented by 
e 2Ti\t an( j foe (complex) amplitude associated with the frequencies be- 
tween X and X -f- dX is denoted by dY\, and hence the contribution of such 
frequencies to the process is 

e 2 * iU dY x . 

Here Y is a process which assigns to each real number X a random variable 
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Y\. The process X is obtained by adding the contributions associated with 
the various frequencies. Hence 






Thus the spectrum of the process X is described by the process Y. 

The expected value of the square of the amplitude associated with the 
frequencies between X and X + dX is denoted by dF(\) and is defined by 

dF(\) = (dY x , dY x ) ^ 0. 

Thus F is a monotone nondecreasing function. A property of the process 
Y, called the property of orthogonal increments, is the following 

(dY x , dY») = 0, 

if the intervals d\, d\x have no common points. Hence 

R(t) = (X t+T , X t ) = (f e 2 **<*+r) dYx, f e 2 ^ 1 dY,\ 

I e 2riX(*+r) e 2™ M * yy^ ^) 

—00 *^— 00 
*'— 00 *^— 00 

As a special case of this formula 

22(0) = f dF(X). 

«^— 00 

The following example of a one-dimensional Brownian motion will aid 
in visualizing a random process. A tiny mirror is suspended by a fiber. 
Particles of air bombard the mirror and cause it to turn through an angle. 
A beam of light is reflected by the mirror and the position of the reflection 
enables the observer to measure the angle X(t) through which the mirror 
has turned at time t. Since X(t) is produced by the average effect of a 
number of bombardments, one might expect X(t) to have a normal distri- 
bution. That is, the probability that X(t) < X is 

1 /* x 

where a 2 is the variance and is assumed to be independent of t. From this 
formula one can readily show that E(X(t)) = 0. The zero angle is the angle 
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in which the fiber is untwisted. If t and t + t are two times at which the 
mirror is observed, the joint probability that X(t) < Xi and X(t -f- r) < X 2 
is 

/ f ' f 2 e -[^-2r(r)x 2/ +^]/2^[l-^(r)] fa j 

livaWi - r 2 {r) ./_ ./_«, 

This is called the bivariate normal distribution. From this formula one 
can show that the covariance is 

(X(t), X(t + r) = a 2 r(r), 

and hence that the process is stationary in the wide sense. If it is known 
that X(t) = a then the probability that X(t + r) < X is 

1 r x 

j e -[x-ar(T)]*l2vnL-rHT)] fa 

c\/2ir(l - r\r)) J^ 

Any information concerning the motion previous to time t is irrelevant to 
this probability. A process having this property is said to be Markovian. 
The assumption that the above process is Markovian implies that 

r( T ) = e~ kT , where k > 0. 
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1. NATURE OF STATISTICS 

The basic assumption underlying the application of the mathematical 
theory of probability and statistics to physical situations is the following: 
If a physical "experiment" is repeated under "identical" conditions and 
"without bias," the observed relative frequency of success of any physical 
"event" approaches as a limit the probability assigned to this event by 
some underlying probability distribution. 

Probability theory is the study of probability distributions as mathe- 
matical entities. Statistics is the analysis of probability distributions on 
the basis of a number of experimental observations; the distribution is 
in general not fully known to start with, and one seeks properties of the 
distribution on the basis of the observations. Since an infinite number 

13-01 
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of experiments would usually be required to determine a distribution with 
precision, it is only rarely possible to answer a statistical question with 100 
per cent surety. Accordingly the answer to each statistical question should 
consist of two parts: (a) the best possible answer to the question and (b) 
the amount of confidence that can be placed in the correctness of this 
answer. The omission of (b) greatly diminishes the value of the conclusion. 

2. PROBABILITY BACKGROUND 

The basic probability theory required for statistics is reviewed in Chap. 
12. For the sake of convenience the principal definitions are recalled here. 
(See Refs. 2, 6.) 

Sample Space. The sample space S is the collection of all possible 
outcomes of a physical experiment; the individual outcomes are sample 
points. By an event is meant a certain type of outcome; in other words, 
a certain set A of sample points. A class G, of events is assumed specified. 
To each event A of class & is assigned a probability, Pr(A), which is a real 
number between and 1. One has Pr(0) = 0, Pr(S) = 1, and Pr(A U B) 
= Pr{A) + Pr(B), provided A, B have no points in common (are mutually 
exclusive events). 

A sample space S is discrete if its points form a finite or infinite sequence 
£i> £2, ' • • • For discrete spaces a probability is usually defined for each 
point, and then for each subset A as the sum of the probabilities of the 
points in A . 

Random Variables. A random variable is a function X = X(£) which 
assigns to each sample point £ a real number x in such a fashion that, for 
each a, the set A for which x 2g a has a probability; thus Pr(X ^ a) is 
well defined. With each random variable X is associated a distribution 
F(x);F(a) = Pr{X ^ a). Fix) isnondecreasing,^(-oo) = 0, F(+ao) = 1. 
If Xi, • • -, X n are random variables associated with the same experiment, 
then their joint distribution is F(xi, • • •, x n ), where F(ai, • • •, a n ) is the 
probability assigned to the set where xi ^ a\, • • •, x n ^ a n . The random 
variable X has a density f, if 

(1) F(x) = f f(t) dt; 

the random variables Xi, • • • , X n have a joint density f if 

(2) F(x u •••,*„) = f ' f •■ f "fit!, "-,t n )dt n ---dh. 

J —00 J —00 " —00 

When the range or collection of values of X forms a discrete sequence 
£1,^2, • • •, then 
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(3) F(a) = E Pr& = **) = E /(**), 

x< go xi ga 

where Pr(X = # t -) = f(xi) is the probability assigned to the set of sample 
points for which X = X{. This can. be generalized to joint distributions. 
Random variables X\, • • •, X n are mutually independent if 

(4) F(x l} • • •, X n ) = Fi(Xi)' ■ -Fnixn), 

where F is the joint distribution and F{(xi) is the distribution of X;. 

Throughout the following it will be assumed that either the range of 
each random variable is discrete or else each distribution has a density 
(continuous case). 

The expectation or mean of a random variable X is 



(5) E(X) = 



I xf(x) dx (continuous case), 
^2xif{xi) (discrete case). 



If is a continuous function of x, then 

I <f>(x)f(x) dx (continuous case), 

J — rrt 



(6) E(4>(X)) = 



11, <$>{%%) f{%i) (discrete case). 



Moments. The moments of X about the origin are the numbers 

(7) A = W), fc=l, 2, ..-. 
The moments of X about the mean are defined by 

(8) Mfc = E((X - M ) fc ), fc = 2,3, •••, 

where ju = E(X) = m'i- The quantity a 2 = n 2 is the variance of X, while 
a- = V a 2 is the standard deviation of X. 

By expanding the quantity (x — ju) fc by the binomial formula and 
applying eq. (8), one obtains an expression for the hj c in terms of n\, • • •, 

Mfc = M'fc — kp'k-ifi -\ • 

In particular, 

(9) a 2 = M ' 2 - 2/xVi + (M'l) 2 = /*' 2 ~ M 2 . 

The mean /* is a measure of the location of the "center" of the distribution, 
while the variance a 2 is a measure of the "spread" of the distribution. 



13-04 GENERAL MATHEMATICS 

Other possible measures of central tendency are: 

Median: a point x such that Pr(X ^ x ) = Pr(X ^ xq), 
Mode: a point x where f(x) is a maximum, 

Midrange: ^{a -\- b), if a ^ x ^ b is the smallest interval containing 
all x for which f(x) > 0. 

Other measures of the spread of the distribution are : 

Mean deviation from the mean = E (\X — n\), 

Probable error: a number a such that Pr{\X — /z| ^ a) = -§-. 

For comparison and tabulating purposes it is useful to describe a random 
variable in a manner independent of origin and scale. These requirements 
are met by the standardized variable X* = (X — /x)/<r, which has mean 0, 
has standard deviation 1, is dimensionless, and is invariant under any 
linear change of variable: X' = aX + b. 

3. IMPORTANT PROBABILITY DISTRIBUTIONS 

Binomial or Bernoulli Distribution. If X represents the number of 
"successes" in n independent trials of an experiment, with probability p 
of "success" each time, then X takes on the values 0, 1, 2, • • • , n with 
probabilities 

(10) f(x) = ( n ) v x ° n ~ x = „ n ' „ V x q n ~ x , ff = 1 - p. 

\x/ x\(n — x)l 

Hence the sample space S has 2 n points £, each representing one particular 
succession of successes and failures. The random variable X assigns to 
each £ the number of successes in £. The mean and standard deviation 
are found to be 

(11) n = np, 

Poisson Distribution. A discrete random variable X with values 
0, 1, 2, • • •, is said to have a Poisson distribution if the corresponding 
function f(x) has form 

X 

(12) f(x)=e-*- (x = 0, 1, 2, ...), 

xl 

where a is a positive constant. One finds 

(13) n = a, <t = v a- 

For large n and small p the binomial distribution (10) is well approximated 
by the distribution (12), with a = np. 
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If a number of events occur independently in space or time and if X 
represents the number of these events occurring in any given space or 
time interval, then the Poisson distribution is a good model for the distribu- 
tion of X. Examples are the number of red corpuscles on a microscope 
slide, the rate of emission of electrons or a-particles, the number of in- 
coming calls to a telephone exchange. 

Normal Distribution. Let X be a continuous random variable with 
density 

(14) ${x) = -j= • e -(*-">W t 

Then X is said to have a normal distribution; its mean and standard deviation 
are /x and a. One terms <t>(x) the normal density function of mean p and 
standard deviation a; the corresponding distribution 

(15) $(3) = I <f>(t) dt 

•^ — 00 

is the normal distribution function. The function $ is tabulated for (j. = 
and o- = 1, and any other case is reduced to this by replacing X by its 
standardized variable X* (Sect. 2). See Table 2, Sect. 9. 

For large values of n, the binomial distribution may be approximated 
by the normal distribution having /z = np, a = 'Vnpq. More precisely, 
if X has a binomial distribution, then as n — » oo 



(16) Pr ( ^ /-'- ^ tj = Pr{X* £t) -> Ht). 



(X — np 
■Vnpq 



The X 2 -Distribution. Let X be a continuous random variable with 
values in the range ^ x < ». Then X is said to have a X 2 -distribution 
with n degrees of freedom, if X has density 

(17) fc n (a;) =—jz x^ nl2) - l e- x ' 2 , x^O. 

K ' 2 n/2 r(n/2) 

One finds 

(18) ix = n, a = V2n (n = 1, 2, •••). 

This type of distribution is of great importance in the theory of sampling 
of normal populations (Sect. 4). See Table 3, Sect. 9. 
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Student t-Distribution. Let X be a continuous random variable 
with density 

f n + V 

,2v-(n + l)/2 



(19) s n (x) = 



Vmr T(n/2) 



0+3' 



Then X is said to have a Student ({-distribution with n degrees of freedom 
(n = 1, 2, • • •)• One finds 



(20) n = 0, 



w - 2 



As n — > co, s n (#) approaches the normal density function of mean and 
standard deviation 1. The ^-distribution is of value in sampling theory 
(Sect. 4). See Table 4, Sect. 9. 

4. SAMPLING 

In a great variety of practical problems a precise answer is obtainable 
only by making a very large number of measurements. For the sake of 
economy, one makes a smaller number of measurements and estimates 
the true answer from these. The theory of such methods of estimation is 
called sampling. .Examples. The average height of 1,000,000 soldiers can 
be estimated by averaging the heights of a selected 1000 soldiers. The 
outcome of a presidential election can be estimated by polling a small 
number of voters. 

The successive measurements in an experiment yield a random sequence 
Xi , • • • , X n called a sample. 

Example. The measurements of the height of 1000 soldiers yield 1000 
numbers. One can regard each soldier as a sample point £, the aggregrate 
of all 1,000,000 soldiers as the sample space S. If the heights follow some 
definite pattern, then there will be a definite probability that the height 
X\ be less than a fixed value. Hence, there is a distribution function F\{x\) 
associated with Xi and X\ can be regarded as the value of a random vari- 
able Xi. Similar statements apply to the measurements X 2 , •••, X n . 
If the measurements are independent (i.e., each one is made without 
considering the others), all measurements have the same distribution 
F(x) and X\, - • • , X n are random variables with joint distribution 

(21) G{x u ••-,*„)= F(x l )F(x 2 ) ■ ■ -F{x n ). 

The assumptions of the example considered will be assumed to hold 
generally. A sample space is assumed given, with associated probabilities. 
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A measurement x is a value of a random variable X; the probability that 
X ^ a is F(a), where F is the distribution of X. Successive measurements 
yield random variables Xi, • • -, X n . It will be assumed that these are 
independent, so that eq. (21) gives the joint distribution. 

Sample Moments. The sample mean or average is the number 

(22) x = 



The sample moments about the origin and about the mean are defined 
respectively as 

in 1 w 

(23) m' k = - £ Xi\ m k = - £ (X { - x) k , 

n i== i n i== i 

so that x = wi'i. The number s 2 = m 2 is the sample variance. One has 
the formula 

(24) s 2 = w' 2 - x 2 . 

One can regard a; and s 2 as estimates for the mean ju and variance a 2 of X; 
x and s 2 , and indeed all the moments, are random variables, being functions 
of X\, • • • , X n . 

From the fact that all X{ have a common distribution F (x), one can 
deduce properties of the distribution of the various moments. For example, 

(25) E(x) = E ( - 2X-) = - 2E(Xi) = - 2 M = M 

\n / n n 

Similarly, 

72/ — 1 

(26) E(s 2 ) = a 2 

n 

Unbiased Estimate. A sample estimate is termed unbiased if its expec- 
tation is equal to the parameter being estimated. Equation (25) shows 
that x is an unbiased estimate of /*; eq. (26) shows that s 2 is not an unbiased 
estimate of a 2 , although [n/(n — l)]s 2 is such an unbiased estimate. 
Unbiasedness is a useful property of an estimate, but it is not as important 
as some other properties. The bias in s 2 need be considered only if n is 
sufficiently small (less than 20, for example), so that (n — V)/n is ap- 
preciably different from 1. 
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Computational Procedures 

Data Classification. The computation of sample moments for large 
samples is simplified by the classification of the data. In this procedure 
the sample range, the interval from the smallest to the largest sample value, 
is divided into approximately fifteen class intervals of equal width (the 
class width). The number of measurements whose X{ value lies in each 
class interval, the frequency of the class interval, is then recorded, as well 
as the midpoint of each interval, the class mark. In the subsequent com- 
putation one then replaces each sample value X{ by the class mark of the 
corresponding class interval ; usually a negligible error is introduced by this 
replacement. Example. In measuring height of a population to the 
nearest 0.1 in. one can choose class intervals 1 in. in width; to avoid am- 
biguity the end points of the class intervals should be 60.05 in., 61.05 in., 
• • • , for instance, rather than 60 in., 61 in., 

Computation. If there are h class intervals with frequencies /y and 
class marks Xj (j = 1, • ■ •, h), then the moments are computed as follows: 

1 * 

(27) * = -Zte-, 

n y=i 

1 * 

(28) w' 2 = - Z fjxj 2 , 

1 h 



(29) s 2 = - £ f&j - x) 2 = ro' 2 - X 



~2 



The computation can be further simplified by coding the data; that is, 
by introducing new measurements y 3 - by a linear change of variables : 

(30) Xj = ayj + 6 (a ^ 0), 

where the coefficients a, b are chosen to simplify the yj data. The new 
mean and variance y and s 2 are related to the old, x and s x 2 , by the equa- 
tions 

'(31) x = ay + b, s 2 = a 2 s y 2 . 

If a is chosen to be the class width and b is taken to be one of the class 
marks (usually chosen near the middle of the range), then the yj are integers, 
positive or negative, so that the computation is considerably simplified. 
After y and s y 2 are computed, x and s x 2 are found from eq. (31). The 
procedure is illustrated in tabular form in Table 1. 
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Table 1. Computation op Sample Mean and Variance 

Coded Marks 



Class 

Intervals 

aj — aj-i = a 



Frequency 

fj 



a - 
a>i- 
a 2 - 
a 3 - 
di- 
a 6 - 
a 7 - 



-Oi 

-a 2 
-a 3 
-at 
-as 
-a 7 
-a & 



II 



h 
I h 



-a 9 



cih-i- 



-a h 



III 



Totals 



n 



™ 2,!/ - y. 



Class Mark 



xi = a + \a 

%2 = «1 + 2« 



& = a 6 + \a 



Xh 



x = ay + 6, 



2A- = 



xy - b 



a 



-5 

-4 

-3 

-2 

-1 



1 

2 



/i2/y 

-10 
-24 



//z/y 2 

50 
96 



ny 

, 2 



nm r 2, v 



Distribution of Sample Moments 

If some information is known concerning the distribution F(x) of the 
random variable X being measured, then one can draw conclusions as to 
the distributions of the sample moments. These conclusions in turn per- 
mit one to make statements as to the accuracy of the sample moments as es- 
timates of the true moments. For example, suppose that the variable X is 
distributed uniformly over an interval of length 1 ; that is, F'(x) = f(x) = 1 
for c ^ x ^ c -f- 1, and/(x) = otherwise. If c is unknown, each sample 
will give information as to its value. A single measurement X then allows 
one to conclude that X — 1 5=j c ^ X, the mean c + 3^ would be esti- 
mated as X and one knows that, with probability 1, the mean lies 
between X — }i and X + Y^. 

One now proceeds to list properties of the distribution of sample moments 
when various assumptions are made concerning the distribution F(x). 
These results are applied below to estimation of accuracy of the estimates. 

Distribution of x When a Is Known. If X is normally distributed, 
then x is also normally distributed, with mean /x and variance a 2 /n (Sect. 3) . 
Equivalently, one can state that 



(32) 



Vn 



has a normal distribution of mean and variance 1. The conclusion is 
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approximately true even if X does not have a normal distribution, provided 
n is large. (See Chap. 12.) 

Distribution of x When a Is Unknown. Let s = v s 2 , the sample 
standard deviation and let 



(33) t 



x — fX / 

= V n - 1. 



so that t can be considered as a random variable. If X is normally dis- 
tributed, then t has a Student ^-distribution with n — 1 degrees of freedom. 
Again the conclusion is approximately true even if X does not have a normal 
distribution, provided n is large. Furthermore, the ^-distribution approaches 
the normal distribution of mean and variance 1 as n — > oo. 
Distribution of s When <r Is Unknown. Let 



(34) u = 



.2 



a 



If X is normally distributed, then u has a x 2 -distribution with n — 1 
degrees of freedom. Again the conclusion is approximately true for large 
n, regardless of the form of F(x). 

Confidence Intervals and Hypothesis Testing 

The results described are now applied to obtain estimates for the accuracy 
of x and s 2 as estimates of n and a 2 . The accuracy will be described in the 
terminology of confidence intervals. The statement "the interval (a, b) 
is a 95 per cent confidence interval for ju" means that Pr(A ^ n ^ B) 
is 0.95, where A, B are random variables with observed values a, b. One 
can also say "either a ^ n ^ b or an event of probability only 0.05 has 
occurred in the sampling." 

Confidence Intervals for ju When <r Is Known. The 95 per 
cent interval is obtained from the fact that (x — n)-\/n/a has a normal 
distribution of mean and variance 1. By means of tables (Sect. 9) one 
determines the number £ .95 on the normal density curve such that 95 
per cent of the area lies between — £ .95 and fo.95> that is, 

(35) Hto.95) - H-to.95) = 0.95. 

(See Fig. 1.) One finds fo.95 = 1-96. Hence with probability 0.95 

-1.96 ^ (x - rfVn/ff ^ 1.96 
or equivalently, 

1.960- 1.96o 

(36) x i=^»^x + — j=r- 

■s/n Vn 
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Thus the 95 per cent confidence interval has end points x ± 1.96 a/\/n. 
In a similar manner one obtains confidence intervals for other percentages. 



$ (x) = area under normal 
curve from -oo to x 




Fig. 1. Normal density curve with 95 per cent limits indicated. 

Confidence Intervals for it When a Is Unknown. Now s must be 
used instead of a and (x — /x) V n — 1/s has a ^-distribution with n — 1 
degrees of freedom. The point fo.95 is obtained from the tables of Sect. 9. 
Then with probability 0.95 



■fo.95 ^ (x — /i)Vri — 1/s ^ to 



95 



or 

(37) 



x — 



3^0.95 

Vn - 1 



^ /x ^ X + 



S^0.95 



V? 



Thus x ± sfo.95/ Vw — 1 are the end points of the 95 per cent confidence 
interval for /x. A similar procedure is used for other percentages. 

Confidence Intervals for a. One uses the fact that ns 2 /a 2 has a 
X 2 -distribution with n — 1 degrees of freedom. Since the x 2 -distribution 



k n -i(x) 




Fig. 2. x 2 -density curve with 95 per cent confidence limits indicated. 

is unsymmetrical, two points £'0.95 and £"0.95 must be found with the aid 
of tables (Sect. 9) such that the areas under the % 2 -density curve to the 
left of i'o.95 and to the right of *" .9 5 both equal 0.025 (Fig. 2). Then with 
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probability 0.95 



or 

(38) 



t 0.95 = T = t 0.95 



ns 



0.95 t 0.95 



Confidence Interval for Binomially Distributed Population. The 

preceding confidence intervals were derived under the assumption that the 
underlying distribution Fix) was normal, or else that n was so large that 
appropriate approximations could be made. Another case of frequent 
occurrence is that for which the distribution is binomial (Sect. 3) with 
unknown probability p of success. The obvious estimate for p is the 
sample proportion of successes x/n. To obtain a confidence interval 
we use the fact that for n large the binomial distribution can be approxi- 
mated by the normal distribution of mean np and variance npq. Hence 
with probability 0.95 

x — np 

— to.95 ^ / ^ fo.95> 



where fo.95 = 1-96, as obtained from Table 2 in Sect. 9. The end points 
of the confidence interval for p are obtained by setting (x — np)/ 
V np (1 — p) equal to ±/o.95 and solving the resulting quadratic equation 
for p. In this way the end points are found as 



c 2 + 2x db cVc + 4x(l - x/n) 



2(c 2 + n) 
or, approximately for large n, as 



c = t 



0.95 



x V x(n — x) 

- ± to.95 U 

n n /2 

Hypothesis Testing. Frequently, instead of obtaining estimates or 
confidence intervals for moments, one is merely interested in answering 
"yes" ■ or "no" to certain hypotheses about the population. For example, 
one is asked, "On the basis of the observed data, are we justified in reject- 
ing the assumption that n has a specified value?" One method of answer- 
ing such a question is to construct a confidence interval, say a 95 per cent 
confidence interval, for ;u. If the specified value lies outside (inside) this 
interval, one replies, "Yes, one is (is not) justified in rejecting the assump- 
tion at the 95 per cent level of significance." See Refs. 2, 4, 5. 
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5. BIVARIATE DISTRIBUTIONS 

Let X and Y be two random variables associated with the same experi- 
ment (that is, denned on the same sample space), and not necessarily 
independent. The following questions arise very frequently: "Is there 
any functional dependence between X and YV "Knowing the value of 
X, how would one best predict the value of Y?" If there is a strict func- 
tional relation Y = <t>(X), then all the values (x, y) fall on the curve 
y = (j)(x). In practice, see Fig. 3, this arises very rarely. Instead, one 



y = $(x) 




Fig. 3. Regression curve of y on x. 



may find the values (x, y) distributed very close to a curve y = <l>(x); that 
is, the density f(x, y) may be very high near the curve and negligible far 
from the curve. For fixed x = t, the values are distributed along the 
line x = t with one -dimensional density, again large near y = 4>{x) and 
small when y differs greatly from y = <j>(x). This one-dimensional density 
is simply the conditional density (Chap. 12) 



(39) 



p(y\t) = 



f fit, 



f(t,y) _ f(t,y) 
fi(t) 



y)dy 



where fi(x),f 2 (y) are the density functions for X and Y. The best estimate 
for <()(x) at x = t is the mean value of Y on the line x = t; that is, the 
value of 



(40) 



M y |x = E(y\x) = I 



yp{y\x)dy. 



Equation (40) defines ii y \ x as a function of x, the true regression function 
of Y on X. 

Computation. If f(x, y) is not completely known, measurements in 
an experiment will lead to various averages from which \x y \ x can be esti- 
mated. If the pairs (Xi, Fi), • • •, (X n , Y n ) are the experimental values 



13-14 GENERAL MATHEMATICS 

(random vectors) then one computes the quantities 

in 1 n 

x = - ^2 Xk, y = - 2Z Yh (sample means) 

n fc=i n fc=i 

(41) 

^ = - i: (x k - x) 2 , s* = -j:(Y k -yY, 

n fc=i n fc=i 

1 n 

(42) s xy = — ^ (Xk — x)(Yk — y) (sample moments of second order), 

ft *=i 

(43) r = (sample coefficient of correlation). 

S X Sy 

These can be considered as estimates for the true means, moments about 
the mean, and coefficient of correlation (Chap. 12) 

(41') fix = I xfi(x) dx, n y = I yf 2 (y) dy, 

J -co J — oo 

(42') a a = I I (x - fi x y(y - fx y ) j f(x, y)dx dy (i, j = 1, 2), 

J — oo J —oo 

°"12 

(43') P = -r^=- 

V o- n a- 2 2 

Computation of the experimental quantities (41), (42), and (43) can be 
simplified by coding techniques analogous to those of Sect. 4. (See Refs. 
3, 8.) 

Curve-Fitting by Least Squares 

From a graphical representation of the data (x k , yh), one is usually led 
to some notion of the form of the regression function n v \ x (mean of Y for 
each X). One then chooses an estimate y. y \ x for this function with one or 
more adjustable parameters. The parameters are then chosen so that 

n 

X) (Vk — fiy\x k ) 2 
k=l 

has its minimum value. This is the method of fitting data by least squares. 
The simplest form for p. y \ x is Ax + B, where A, B are adjustable. The 
method of least squares leads to the values 

y y 

(44) A = — . B = y — rx—- 

s x s x 

More generally, one can let 

(45) jly lx = A Q + Aix -] h A m x m , 
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where the constants Aq, ■••, A m are adjustable. The method of least 
squares leads to the following linear equations for Aq, • • •, A m : 

in n n 

(46) £ £ A&j+i = E aj^fc (j = 0, 1, • • -, m). 

i=0 fc=l fc=l 

Confidence Intervals 

In order to obtain confidence intervals for the estimates, it is necessary 
to make certain assumptions concerning the form of the underlying density 
f(x, y). Let us assume that the true regression function ii y \ x is linear, and 
that the distribution of Y for fixed X is normal, with variance independent 
of x. These assumptions are often satisfied in practical problems. In 
practice, the variable X is usually not determined by random samples, 
but it is given preassigned values and the corresponding values of Y are 
determined experimentally. (Example. In a problem of detection of 
electromagnetic radiation, X might denote the range and Y the signal 
strength; the range X would be varied at regular intervals and the signal 
strength Y recorded once or several times for each value of X.) Under 
these assumptions, the true regression function is of form ax + b. The 
coefficients a and b are estimated by A, B as given in eq. (44). The two 
quantities for which confidence intervals are usually required are n y \ x 
itself and the regression coefficient a. 

The required intervals are obtained as follows. One defines 

(47) S 2 = - £ (y k - Ax k - B)* = s y \l - r 2 ). 

n k =i 

It can then be shown that the quantities 

(ji u \ x — Ax — B)-s x - y/n — 2 (a — A)s x Vn — 2 



(48) 



V(x-x) 2 + s x 2 -S S 



possess f-distributions with n — 2 degrees of freedom. Thus 95 per cent 
confidence intervals for ii y \ x and a have end points 

V(x- x) 2 + s x 2 -S 

Ax + B ± fo.95 / ' 

s x \/n — 2 

m 

A ± fo.95 / 

s x Vn — 2 

respectively, where £ .95 is determined as in Sect. 4 from the ^-distribution 
with n — 2 degrees of freedom. 

The preceding process can be generalized to the case of more than two 
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variables. For details on .this and on computational procedures, see 
Refs. 3, 5, 8. 

6. TESTS FOR GOODNESS OF FIT 

A problem of frequent occurrence is to determine whether a set of 
experimentally determined data is consistent with, or fits, some pre- 
assigned hypothetical probability distribution. The usual way of studying 
this is to divide the range of the variable X into k subintervals, not neces- 
sarily equal in length. The number /; of observations falling in the ith sub- 
interval can then be counted. Let pi be the probability assigned to the 
ith subinterval by the given hypothetical probability distribution. Let 
n be the total number of observations. It can then be shown that the 
quantity 

(50) u = 2^ 

*=i npi 

has approximately a % 2 -distribution with n — 1 degrees of freedom. From 
eq. (50) it is apparent that large values of u correspond to large differences 
between the observed and theoretical distributions. If it is greater than the 
0.95 point of the x 2 -table, one can say (with 95 per cent certainty) that the 
sample did not come from the given hypothetical distribution. 

The approximation by a x 2 -distribution is usually sufficiently accurate 
if each pi > 5/n and k > 5. 

Frequently in such a problem the hypothetical distribution is not 
completely specified, but contains some adjustable parameters. For 
example, one might wish to test whether a sample comes from a normal 
population, in which case the mean and variance of the population must 
first be estimated from the sample. It can be shown that the x 2 -test 
usually remains valid, provided one further degree of freedom is sub- 
tracted for each parameter estimated. More precisely, in order for the 
test to be valid, the parameters must be estimated by the method of maxi- 
mum likelihood. See Refs. 4, 5. 

7. SEQUENTIAL ANALYSIS 

The usual method of collecting data consists of the determination of a 
fixed number of observations and their subsequent statistical analysis. 
Frequently a considerable reduction in the number of observations re- 
quired can be made by making the observations in sequence and re- 
analyzing the data after each observation. Such a process is known as a 
sequential analysis and is particularly useful for such problems as production 
testing. 

Example. Consider a population whose density function f(x; 8) depends 
on some parameter 6 (mean, variance, etc.) whose value is not known; let 
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us suppose that can take only one of two given values O , 0i- The prob- 
lem is to decide which value is the correct one. In such a decision problem, 
errors can be made in two ways: by deciding that 0i is correct when O 
is actually the true value of 0, or by deciding that O is correct when 0i 
is actually the true value. Denote the probabilities to be assigned to 
these two types of errors by a and j3 respectively. The values of a and /3 
can be preassigned by an experimenter, and clearly both should be small 
if one wants to have great confidence in one's decision; however, the 
smaller a and j8 are taken to be the more observations will be required to 
come to a decision. 

Let Xi, x 2 , • • • be the sequence of observed values, and let f(x; 6/) denote 
the density function of the population when 8j is the true value of 6, j = 
0,1. Define the quantities 

n -p 

(51) P in = II /(**; Oj) (i = o, 1), Qn=~ 

i=\ * On 

Each Pj n can be found from the preceding one after each observation by 
multiplying by the corresponding f(x n ; df). The decision rule is then the 
following. 
If 

|8 1-0 

(52) < q n < 

1 — a a 

take another observation. If q n ^ /3/(l — a), decide that O is the correct 
value of 0; if q n ^ (1 — j8)/a, decide that 0i is the correct value of 0. 
Thus the rule is to continue sampling until q n leaves the interval (52), 
choosing O if q n first leaves the interval to the left and choosing t if q n 
first leaves the interval to the right. 

A formula is available which allows one to estimate the mean number 
of observations required before a decision is reached. In general, this 
number is substantially less than the number required when the sample 
size is fixed, although one must expect as many or more observations to 
be required in a small percentage of cases. (See Ref. 7.) 

8. MONTE CARLO METHOD 

A great variety of mathematical and physical problems of apparently 
nonstatistical nature can be reformulated in statistical form and solved 
by sampling techniques. For example, the area under a curve y = f(x), 
^ x ^ 1, ^ f(x) ^ 1 can be formulated as the probability that a 
"random point" (x, y) in the square O^rc^l, 0^2/^1 satisfies the 

inequality y ^ fix) . Thus computation of I f(x) dx is achieved by choos- 
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ing many random points (x, y) and counting the proportion which satisfy 
the condition y ^ /(#). 

, The general program of such statistical solutions to essentially non- 
statistical problems is described as the Monte Carlo method. The basic 
idea is very old, but the method has been the subject of unusual interest 
in the last decade, especially because of the availability of high-speed 
digital computing machines. 

For information on the subject see Ref. 9, which contains an extensive 
bibliography. 

9. STATISTICAL TABLES 



Table 2. The Cumulative Normal Distribution Function (Ref. 10) 

*(u) = —1= I e 2 dx for 0.00 g u g 2.99. 



u 


.00 


.01 


.02 


.03 


.04 


.05 


.06 


.07 


.08 


.09 


.0 


.5000 


.5040 


.5080 


.5120 


.5160 


.5199 


.5239 


.5279 


.5319 


.5359 


.1 


.5398 


.5438 


.5478 


.5517 


.5557 


.5596 


.5636 


.5675 


.5714 


.5753 


.2 


.5793 


.5832 


.5871 


.5910 


.5948 


.5987 


.6026 


.6064 


.6103 


.6141 


.3 


.6179 


.6217 


.6255 


.6293 


.6331 


.6368 


.6406 


.6443 


.6480 


.6517 


.4 


.6554 


.6591 


.6628 


.6664 


.6700 


.6736 


.6772 


.6808 


.6844 


.6879 


.5 


.6915 


.6950 


.6985 


.7019 


.7054 


,7088 


.7123 


.7157 


.7190 


.7224 


.6 


.7257 


.7291 


.7324 


.7357 


.7389 


.7422 


.7454 


.7486 


.7517 


.7549 


.7 


.7580 


.7611 


.7642 


.7673 


.7703 


.7734 


.7764 


.7794 


.7823 


.7852 


.8 


.7881 


.7910 


.7939 


.7967 


.7995 


.8023 


.8051 


.8078 


.8106 


.8133 


.9 


.8159 


.8186 


.8212 


.8238 


.8264 


.8289 


.8315 


.8340 


.8365 


.8389 


1.0 


.8413 


.8438 


.8461 


.8485 


.8508 


.8531 


.8554 


.8577 


.8599 


.8621 


1.1 


.8643 


.8665 


.8686 


.8708 


.8729 


.8749 


.8770 


.8790 


.8810 


.8830 


1.2 


.8849 


.8869 


.8888 


.8907 


.8925 


.8944 


.8962 


.8980 


.8997 


.90147 


1.3 


.90320 


.90490 


.90658 


.90824 


.90988 


.91149 


.91309 


.91466 


.91621 


.91774 


1.4 


.91924 


.92073 


.92220 


.92364 


.92507 


.92647 


.92785 


.92922 


.93056 


.93189 


1.5 


.93319 


.93448 


.93574 


.93699 


.93822 


.93943 


.94032 


.94179 


.94295 


.94408 


1.6 


.94520 


.94630 


.94738 


.94845 


.94950 


.95053 


.95154 


.95254 


.95352 


.95449 


1.7 


.95543 


.95637 


.95728 


.95818 


.95907 


.95994 


.96080 


.96164 


.96246 


.96327 


1.8 


.96407 


.96485 


.96562 


.96638 


.96712 


.96784 


.93856 


.96926 


.96995 


.97062 


1.9 


.97128 


.97193 


.97257 


.97320 


.97381 


.97441 


.97500 


.97558 


.97615 


.97670 


2.0 


.97725 


.97778 


.97831 


.97882 


.97932 


.97982 


.98030 


.98077 


.98124 


.98169 


2.1 


.98214 


.98257 


.98300 


.98341 


.98382 


.98422 


.98461 


.98500 


.98537 


.98574 


2.2 


.98610 


.98645 


.98679 


.98713 


.98745 


.98778 


.98809 


.98840 


.98870 


.98899 


2.3 


.98928 


.98956 


.98983 


.9 2 0097 


.9 2 0358 


.9 2 0613 


.9 2 0863 


.9 2 1106 


.9 2 1344 


.9 2 1576 


2.4 


.9 2 1802 


.9 2 2024 


.9 2 2240 


.9 2 2451 


.9 2 2656 


.9 2 2857 


.9 2 3053 


.9 2 3244 


.9 2 3431 


.9 2 3613 


2.5 


.9 2 3790 


.9 2 3963 


.9 2 4132 


.9 2 4297 


.9 2 4457 


.9 2 4614 


.9 2 4766 


.9 2 4915 


.9 2 5060 


.9 2 5201 


2.6 


.9 2 5339 


.9 2 5473 


.9 2 5604 


.9 2 5731 


.9 2 5855 


.9 2 5975 


.9 2 6093 


.9 2 6207 


.9 2 6319 


.9 2 6427 


2.7 


.9 2 6533 


.9 2 6636 


.9 2 6736 


.9 2 6833 


.9 2 6928 


.9 2 7020 


.9 2 7110 


.9 2 7197 


.9 2 7282 


.9 2 7365 


2.8 


.9 2 7445 


.9 2 7523 


.9 2 7599 


.9 2 7673 


.9 2 7744 


.9 2 7814 


.9 2 7882 


.9 2 7948 


.9 2 8012 


.9 2 8074 


2.9 


.9 2 8134 


.9 2 8193 


.9 2 8250 


.9 2 8305 


.9 2 8359 


.9 2 8411 


.9 2 8462 


.9 2 8511 


.9 2 8559 


.9 2 8605 



Example: *(2.57) = .9 2 4915 = .994915. 



Table 3. The x 2 Distribution 
P = the probability of a x 2 deviation greater than the tabulated value 




Degrees 




























of 


P =0.99 


0.98 


0.95 


0.90 


0.80 


0.70 


0.50 


0.30 


0.20 


0.10 


0.05 


0.02 


0.01 


freedom 




























1 


0.000157 


0.000628 


0.00393 


0.0158 


0.0642 


0.148 


0.455 


1.074 


1.642 


2.706 


3.841 


5.412 


6.633 


2 


0.0201 


0.0404 


0.103 


0.211 


0.446 


0.713 


1.386 


2.408 


3.219 


4.605 


5.991 


7.824 


9.210 


3 


0.115 


0.185 


0.352 


0.584 


1.005 


" 1.424 


2.366 


3.665 


4.642 


6.251 


7.815 


9.837 


11.341 


4 


0.297 


0.429 


0.711 


1.064 


1.649 


2.195 


3.357 


4.878 


5.989 


7.779 


9.488 


11.668 


13.277 


5 


0.554 


0.752 


1.145 


1.610 


2.343 


3.000 


4.351 


6.064 


7.289 


9.236 


11.070 


13.388 


15.086 


6 


0.872 


1.134 


1.635 


2.204 


3.070 


3.828 


5.348 


7.231 


8.558 


10.645 


12.592 


15.033 


16.812 


7 


1.239 


1.564 


2.167 


2.833 


3.822 


4.671 


6.346 


8.383 


9.803 


12.017 


14.067 


16.622 


18.475 


8 


1.G46 


2.032 


2.733 


3.490 


4.594 


5.527 


7.344 


9.524 


11.030 


13.362 


15.507 


18.168 


20.090 


9 


2.088 


2.532 


3.325 


4.168 


5.380 


6.393 


8.343 


10.656 


12.242 


14.684 


16.919 


19.679 


21.666 


10 


2.558 


3.059 


3.940 


4.865 


6.179 


7.267 


9.342 


11.781 


13.442 


15.987 


18.307 


21.161 


23.209 


11 


3.053 


3.609 


4.575 


5.578 


6.989 


8.148 


10.341 


12.899 


14.631 


17.275 


19.675 


22.618 


24.725 


12 


3.571 


4.178 


5.226 


6.304 


7.807 


9.034 


11.340 


14.011 


15.812 


18.549 


21.026 


24.054 


26.217 


13 


4.107 


4.765 


5.892 


7.042 


8.634 


9.926 


12.340 


15.119 


16.985 


19.812 


22.362 


25.472 


27.688 


14 


4.660 


5.368 


6.571 


7.790 


9.467 


10.821 


13.339 


16.222 


18.151 


21.064 


23.685 


26.873 


29.141 


15 


5.229 


5.985 


7.261 


8.547 


10.307 


11.721 


14.339 


17.322 


19.311 


22.307 


24.996 


28.259 


30.578 


16 


5.812 


6.614 


7.962 


9.312 


11.152 


12.624 


15.338 


18.418 


20.465 


23.542 


26.296 


29.633 


32.000 


17 


6.408 


7.255 


8.672 


10.085 


12.002 


13.531 


16.338 


19.511 


21.615 


24.769 


27.587 


30.995 


33.409 


18 


7.015 


7.906 


9.390 


10.865 


12.857 


14.440 


17.338 


20.601 


22.760 


25.989 


28.869 


32.346 


34.805 


19 


7.633 


8.567 


10.117 


11.651 


13.716 


15.352 


18.338 


21.689 


23.900 


27.204 


30.144 


33.687 


36.191 


20 


8.260 


9.237 


10.851 


12.443 


14.578 


16.266 


19.337 


22.775 


25.038 


28.412 


31.410 


35.020 


37.566 


21 


8.897 


9.915 


11.591 


13.240 


15.445 


17.182 


20.337 


23.858 


26.171 


29.615 


32.671 


36.343 


38.932 


22 


9.542 


10.600 


12.338 


14.041 


16.314 


18.101 


21.337 


24.939 


27.301 


30.813 


33.924 


37.659 


40.289 


23 


10.196 


11.293 


13.091 


14.848 


17.187 


19.021 


22.337 


26.018 


28.429 


32.007 


35.172 


38.968 


41.638 


24 


10.S56 


11.992 


13.848 


15.659 


18.062 


19.943 


23.337 


27.096 


29.553 


33.196 


36.415 


40.270 


42.980 


25 


11.524 


12.697 


14.611 


16.473 


18.940 


20.867 


24.337 


28.172 


30.675 


34.382 


37.652 


41.566 


44.314 


26 


12.198 


13.409 


15.379 


17.292 


19.820 


21.792 


25.336 


29.246 


31.795 


35.563 


38.885 


42.856 


45.642 


27 


12.879 


14.125 


16.151 


18.114 


20.703 


22.719 


26.336 


30.319 


32.912 


36.741 


40.113 


44.140 


46.963 


28 


13.565 


14.847 


16.928 


18.939 


21.588 


23.647 


27.336 


31.391 


34.027 


37.916 


41.337 


45.419 


48.273 


29 


14.256 


15.574 


17.708 


19.768 


22.475 


24.577 


28.336 


32.461 


35.139 


39.087 


42.557 


46.693 


49.5SS 


30 


14.953 


16.306 


18.493 


20.599 


23.364 


25.508 


29.336 


33.530 


36.250 


40.256 


43.773 


47.962 


50.892 



For degrees of freedom greater than 30, the expression v'jjx* — V2n' — 1 may be used as a normal deviate with unit variance, where n' ia the number of degrees 
Reproduced from Statistical Methods /or Research Workers, 6th ed., with the permission of the author, R. A. Fisher, and his publisher, Oliver and Boyd, Edinburgh. 
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Table 4. Student's t Distribution * 



Degrees 

of 

freedom n 




Probability of a deviation greater than t 




.005 


.01 


.025 


.05 


.1 


.15 


1 


63.657 


31.821 


12.706 


6.314 


3.078 


1.963 


2 


9.925 


6.965 


4.303 


2.920 


1.886 


1.386 


3 


5.841 


4.541 


3.182 


2.353 


1.638 


1.250 


4 


4.604 


3.747 


2.776 


2.132 


1.533 


1.190 


6 


4.032 


3.365 


2.571 


2.015 


1.476 


1.156 


6 


3.707 


3.143 


2.447 


1.943 


1.440 


1.134 


7 


3.499 


2.998 


2.365 


1.895 


1.415 


1.119 


8 


3.355 


2.896 


2.306 


1.860 


1.397 


1.108 


9 


3.250 


2.821 


2.262 


1.833 


1.383 


1.100 


10 


3.169 


2.764 


2.228 


1.812 


1.372 


1.093 


11 


3.106 


2.718 


2.201 


1.796 


1.363 


1.088 


12 


3.055 


2.681 


2.179 


1.782 


1.356 


1.083 


13 


3.012 


2.650 


2.160 


1.771 


1.350 


1.079 


14 


2.977 


2.624 


2.145 


1.761 


1.345 


1.076 


15 


2.947 


2.602 


2.131 


1.753 


1.341 


1.074 


16 


2.921 


2.583 


2.120 


1.746 


1.337 


1.071 


17 


2.898 


2.567 


2.110 


1.740 


1.333 


1.069 


18 


2.878 


2.552 


2.101 


1.734 


1.330 


1.067 


19 


2.861 


2.539 


2.093 


1.729 


1.328 


1.066 


20 


2.845 


2.528 


2.086 


1.725' 


1.325 


1.064 


21 


2.831 


2.518 


2.080 


1.721 


1.323 


1.063 


22 


2.819 


2.508 


2.074 


1.717 


1.321 


1.061 


23 


2.807 


2.500 


2.069 


1.714 


1.319 


1.060 


24 


2.797 


2 492 


2.064 


1.711 


1.318 


1.059 


25 


2.787 


2.485 


2.060 


1.708 


1.316 


1.058 


26 


2.779 


2.479 


2 056 


1.706 


1.315 


1.058 


27 


2.771 


2.473 


2.052 


1.703 


1.314 


1.057 


28 


2.763 


2.467 


2.048 


1.701 


1.313 


1.056 


29 


2.756 


2.462 


2.045 


1.699 


1.311 


1.055 


30 


2.750 


2.457 


2.042 


1.697 


1.310 


1 055 


oo 


2.576 


2.326 


1.960 


1 645 


1.282 


1.036 



The probability of a deviation numerically greater than t is twice the 
probability given at the head of the table. 

* This table is reproduced from Statistical Methods for Research Workers, with the generous 
permission of the author, Professor R. A. Fisher, and the publishers, Messrs. Oliver and Boyd. 
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1. INTERPOLATION, CURVE FITTING, DIFFERENTIATION, 

AND INTEGRATION Bernard Dimsdale 

Definitions. Suppose f(x) is a function about which the following is 
known: at each of n + 1 points x , x\, • • • , x n , called the basic set of points, 
the numerical value of / or of one of its derivatives is known. It is to be 
noted that x may represent one or more independent variables. Suppose 
g(x; a , a^ • • •, a n ) is given analytically and the a's are determined so 
that g has the same numerical property as / at each point of the basic 
set. Then g is called an interpolating function for/, and R = f — g is called 
the remainder. 

14-01 
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In the event that g is linear with respect to the a's, that is g(x; a) = 
aogo(%) ~\- aigi(,x) -\- • • • -\- a n g n (x) the interpolating function is called 
linear, and the functions go, gi, • • •, g n are called basic interpolating func- 
tions. In the further event that a; is a single variable and gi(x) = x 1 the 
function g is called an interpolating polynomial. 

If a function g(x; a , • • • , a m ) is given analytically for m ^ n, any require- 
ment whatsoever on / — g over the basic set establishes g as a curve-fitting 
function. If that requirement is that 

n 

Z) if( x i) - Q( x i) a)] 2 w(xi) 

be minimal then g is a least square fit to /, relative to the weight function w, 
which is presumed to be positive. Again g may be nonlinear, linear, or 
polynomial. 

Interpolation 

General Solution of Interpolating Problem. For nonlinear g the 
definitions imply that the a's can be determined by solving n + 1 simul- 
taneous nonlinear algebraic equations. For linear g the equations for a 
are linear and the problem is solved when an (n -\- l)st order matrix is 
inverted, which of course presupposes that it is not singular. No element 
of this matrix depends on the values of / or its derivatives, so that the in- 
verted matrix can be used for all those functions / for which the conditions 
of interpolation, the basic interpolating functions, and the basic set of 
points are the same. 

Interpolating Polynomials for Arbitrary Basic Point Sets. If the 
derivatives of/ are not involved in the interpolation, then 

A K(x) 
g(x) = 1^ rr~\^ Xi ^ 

i=0 fli{Xi) 



R(x) = 



f^ n+i) m(x) 

(n+l)! 



where h(x) is the product of all x — x v , v = 0, 1, • • • , n; hi(x) is the same 
except that!'the factor x — X{ is deleted, / (n+1) (£) is the (n + l)st deriva- 
tive of f(x), £ is an unknown function of x, but is some number between 
the least and the greatest of the basic set of points. This is Lagrange's 
formula, and has been put in practicable computing form by Aitken. Form 
the table 
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Xq /o.o 

X\ /l,0 /l.l 

X2 h,o h,l fl,2 

Xz jzfi /3.1 /3,2 /3,3 



where // >0 = f(xj), 

, , . fa ~ x )(fjJ~fk,j) 
Jk,j+i = fjj ^ ' J > 0. 

Xk Xj 

If sufficient information about some derivative, say the pth, is available 
to show that R = f ( - p) (^)h(x)/p\ is sufficiently small for interpolation 
purposes, p values of x and / will give the interpolated value of / as the 
rightmost number in the table, within an error bounded by R. If not, 
then an iterative process can be undertaken, starting with two points and 
adding one point at a time together with entries in the table until two 
consecutive values in the next to the last column are sufficiently close. 
Example, x = 0.4. 

k Xk flc.O fk,l fk,2 fk,3 flc.4 



1 1 1 0.4 

2 2 8 1.0 -0.32 

3 0.5 0.125 0.1 0.04 0.0G4 

4 -1 -1 0.4 0.4 0.064 0.0G4 

Hence /(0.4) is taken to be 0.0G4. Here f(x) = x 3 , / (4) (x) = 0. Thus 
four points would have been sufficient. 

In the event that derivatives are also given, Neville's procedure applies 
(seeRef. 1). 

Interpolating Polynomials for Uniformly Spaced Points. In the 
event that the basic set of points has the property that x p+i — x p = h, 
where h does not change with p, the procedure to be followed, if derivatives 
do not enter, involves a difference table as follows: 



Xp 


u 


A/p 


A%-i 


A 3 / P _i 


Xp+l 


fp+l 


A/p+i 


A% 


A 3 / P 


Xp+2 


fp+2 


A/ P +2 


A%+i 


A 3 /p+i 


Xp+3 


fp+3 




A%+2 





where A k f q = A k %+i — A* ! /<? and// = /(#/), that is, any element with 
a A is the difference of its two adjacent left neighbors, and is obtained by 
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subtracting the upper one from the lower one, and the subscripts on / are 
constant along a line running diagonally downward to the right. 
Let 

X — Xq 
u = 



(U) r = 



h 
u(u — 1) • • • (w — r + 1) 



r! 
then 

gfr) =f P +(u- p)iA/ p + (w - p) 2 A 2 / p H h (u ^ p)jfcA*/ p , 

and 

«(x)=/ ( * +1) (f)fc* +1 (t*-p)* + i, 

which is Newton's forward formula over the basic set of points 

Xp, £p_|_i, ' ' *, X p _|_fc. 

There is just one interpolation polynomial through these points, but it 
has many representations in terms of differences. In fact, there is a 
representation involving the set of differences obtained by starting with 
the highest order difference, adding to the set one of its two left neighbors, 
and repeating until the / column is reached. The precise representation 
can be written down by use of a Lozenge diagram (see Ref. 2 or 3). Since 
all these are representations of the same interpolation polynomial, the 
remainder term is the same for all of them, and depends only on the par- 
ticular highest order difference used. 

The Newton Interpolating Formulas. With the notation [x] = 
largest integer in x, so that, for example, [2] = [2.2] = [5/2] = 2, the 
following four formulas serve three purposes: 

(a) Newton forward (basic set x , x\, •••,rc n ): 

n 

m =/ + E ^) P ^ p fo + f (n+1) m n+1 (y)n+i. 

p=l 

(b) Newton backward (basic set x_ n , • • •, £_i, x ): 

fix) =f +i:(u + P - l) p A p f- p +f n+1 HZ)h n+ \u + n) B+1 . 
p=l 

(c) Newton-Gauss 1 (basic set X— v , x\— v , ■ • •, x n _ v ; v = [n/2]): 

fix) =fo+T,(u + [p/2 - h]) P A p f- [Pl2] + / (B+1) tt)*" +1 (" + [n/2]) B+1 . 
p=i 
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(d) Newton-Gauss 2 (basic set x_ v , X\_ v , • • •, x n _ v \ v = [n/2 -f- ^]): 

n 

/(*)=/o + E (« + [p/2])pA"/_ [l , /a+w] 

+ / (n+1) (^ +1 (u + N/2 4- J]) B+1 . 

All the £'s, of course, are different and have values somewhere in the interval 
defined by the basic set of points. 
Example. Find /(3. 4256). 



X 


/(*) 


/ 


A 2 / 


A 3 / 


3.0 


0.4771213 


0.0280287 






3.2 


0.5051500 


0.0263289 


-0.0016998 


0.0001945 


3.4 


0.5314789 


0.0248236 


-0.0015053 


0.0001628 


3.6 


0.5563025 


0.0234811 


-0.0013425 


0.0001378 


3.8 


0.5797836 


0.0222764 


-0.0012047 




4.0 


0.6020600 









by using 

(a) Newton forward, x = 3.4. 

( u ) l = u = 0.128, (u) 2 = -0.055808, (u) 3 = +0.05224, 

f(x) = 0.5314789 + 0.0248236(2*)! - 0.0013425(w) 2 + 0.0001378(u) 3 
= 0.5347384. 

(b) Newton backward, x = 3.6. 

(u) x = u = -0.872, (u + 1) 2 = -0.055808, (u + 2) 3 = -0.02098, 

/(a;) = 0.5563025 + 0.0248236(u)i - 0.0015053 (u + 1) 2 

+ 0.0001945(m + 2) 3 
= 0.5347362. 

(c) Newton-Gauss 1, # = 3.4. 

( u ) 1 = u = 0.128, (u) 2 = -0.055808, (u + 1) 3 = -0.02098, 

f(x) = 0.5314789 + 0.0248236(w)i - 0.0015053 (u) 2 + 0.0001628(w + 1) 3 

= 0.5347369. 
Here f(x) is logi z and f(x) is in fact 0.5347366. 
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In interpolating for f(x) it is reasonable to select a formula such that 
x , X\ (or x , x—i) contains x, for then the contribution of the differences 
to the computed f(x) will in general be smallest, a point of considerable 
interest if remainders cannot be computed. If this condition is accepted, 
then forward differences must be used at the beginning of a table or im- 
mediately following a discontinuity in / or its pertinent derivatives. Like- 
wise backward differences must be used at the end of a table or preceding 
a discontinuity. Centered differences may be used in the body of the 
table. In this connection a further comment should be made. The func- 
tion (u) n+ i takes on much smaller values on the average near the center 
of its basic set of points than near the edge. Using centered differences will 
therefore normally achieve a given accuracy with less differences. 

Curve Fitting 

Linear Least Square Curve Fitting. Upon differentiating 

n 

S [aoQoM + aigi(x a ) -\ \- a m g m {x s ) — f(x s )] 2 w(x s ), 

s=0 

with respect to each of the a's, and setting these derivatives equal to zero, 
the equations 

GkodO + Gkldi -\ h Gkrrflm = F k , k = 0, 1, • • ■ , M 

result, where 

n 

Gn = 2 giMgj{xs)w{x a ), 

s=0 

n 

Fi = Yj Qi(xs)f(xs)w(x s ), 

s=0 

thus leading again to a linear system of equations to be solved. Note 
that the matrix G is the product of a rectangular matrix whose elements 
are giixj) by its transpose and is therefore a non-negative definite sym- 
metric matrix. Symmetry implies that the amount of labor in solving 
the linear system is half the normal amount, and non-negative definiteness 
implies that any indication of singularity obtained means that the system 
is singular, which is not necessarily true in the more general case. Note 
that nothing up to this point prevents x from representing more than one 
independent variable, so that surfaces in many variables are amenable to 
this kind of fit. 

The task of forming the G matrix will greatly exceed the task of inverting 
if n is much greater than m, since the number of multiplications (and 
additions) is measured by ytynn 2 , whereas the corresponding number for the 
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inversion is %m?. If, however, gk(x) is x k , e kx , or e kx ^~ l , then Gij = G rs 
if i + j = r + s. Thus only 2?i -+- 1 of these numbers need be computed, 
at a cost of about mn multiplications and additions. That is, linear least 
square fitting with polynomials in z, in e x , or with trigonometric sums in- 
volving terms such as sin kx, cos kx is much less laborious than the more 
general case. 

Machine Solution for Linear Fitting. In any but the simplest cases, 
for very limited values of m and n, this kind of calculation is fit material 
for an automatic computer. If subroutines for evaluation of <7&(x), for 
the product of two rectangular matrices and for inversion of positive 
definite symmetric matrices are available, it is a relatively simple matter 
to program the solution of the problem, including evaluation of residuals, 
that is, of 

n 

2 a s g s (x k ) - f(x k ), 

s=0 

for all k. 

The technique for polynomial least square fitting has been reworked by 
von Holdt (Ref . 4) so that any given set of data (one independent variable) 
can be fitted with polynomials of every order from 1 to m, and all residuals 
computed for each polynomial, with a total computing labor no greater 
than that required for direct fitting, and calculation of residuals, of an ?nth 
order polynomial. With such a procedure it is possible, for example, to 
continue the computation until a polynomial is reached for which the 
largest absolute residual is sufficiently small. There is no doubt that this 
is possible, barring singularities, since for m = n every residual is zero. 

The method of von Holdt loses its value for large values of n, because 
there is a rapid accumulation of roundoff error. This difficulty is mini- 
mized by a method which makes use of orthogonal polynomials (Ref. 4a). 

Nonlinear Least Square Curve Fitting. The problem of selecting 
a = (a , a\, - - -, a m ) so as to minimize 

n 

T{a) = X) fafea) - f(x s )] 2 w(x s ) 

s=0 

may be approached by Newton's method as follows. Let a — a °, ai°, 
• • • , a m ° be arbitrarily assigned and compute 

» dg(x s ; a ) dg(x s ; a°) 



" n dg(x s ; a ) 

s=0 ° a i 
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the notation dg(x s ; a°)/da{ meaning that g is differentiated partially with 
respect to a z - and a is then set equal to a , which is a set of known numbers. 
Solve the system of equations 



J2 On Aoy = Fi, i = 0, 1, 



m, 



3=0 



for Acij and let 

a/ = a/ + Aay, j = 0, 1, • • •, m. 

With a/ as a new a®, iterate the process, which may converge to a solution 
of the problem or, more precisely, to a relative minimum of T. The G 
matrix here is again non-negative definite. 

Levenberg's Method. Unfortunately this process may not converge. 
Levenberg (Ref. 5) has provided a modification which guarantees that 
each step of the process reduces the value of T unless T is already at a 
relative minimum. The iterations go as follows: 

1. Compute the G matrix as before. If G has no zeros on the diagonal, 
call it the current G matrix. If it has, replace them by arbitrary positive 
numbers, for which neighboring nonzero diagonal elements will suffice, 
and call this the current G matrix. No damage is suffered if small but 
nonzero elements are replaced in this way. 

2. If the current G matrix is singular, or effectively singular for practical 
purposes, go to (4). If not, compute Aa and hence a 1 , by using the current 
G matrix. 

3. Compute T{a l ). If T(a x ) < T(a°), let a 1 become the new a and go 
to (1). If TV) ^ T(a°), go to (4). 

4. Double the diagonal elements of the current G to form a new current 
G and go to (2). 

The description is complete, except for a choice of criterion on which 
to terminate the process. The criterion may involve specifying an ac- 
ceptable value of T, an acceptable value of the maximum residual, or a 
maximum number of doublings of the diagonal, for example. 

In practice, it develops that for most iterations no replacing of diagonal 
elements and no doubling are necessary. Such iterative steps are identical 
with Newton's method. Only when Newton's method fails does the ad- 
ditional procedure become operative. Therefore, speaking loosely, Leven- 
berg's method resembles Newton's method as closely as possible without 
being subject to the possibility of divergence. 

Differentiation 

Forward and Backward Formulas for Numerical Differentiation. 

The most commonly used formulas are those obtained by differentiating 
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the Newton formulas (see subsection, The Newton Interpolating For- 
mulas) for equally spaced arguments. For Newton's forward difference 
formula the result is 

n 

hrf m \x ) = £ &»,. a s / + ft B+ V„+i/ (n+1) (& 

S=Jrt 

where / (m) (x ) is the rath derivative of f{x) at a; = x and Table 1 gives 
values of b m>s adequate for finding derivatives up to and including order 
6 by interpolating over as many as eight points. 
For Newton's backward formula the result is 

n 

tf7 (w) (*o) = £ (-1) S+ X, S A 8 /- s + (-l) n+m+1 ^ +1 & m ,» + i/ (B+1) (a 

Again, Table 1 gives the values of b. 

Table 1. Coefficients for Newton's Difference Formulas 



3,s «4,s 05,s 



s &i,„ h, s 

1 1 

2 -i 1 

3 1-11 

4 -i H "f 1 

5^ 6 4 4 1 

ft _JL 13 7 J_5 J_7 _5_ 1 

u 6180 8 6 2 A 

7 1 _7_ _2 7. 2 5 Q 

' T 10 15 2 ~6~~ 

Q 1 3JL3. 4Ci) 967 Ji_5 23 

° "8 560 240 240 6" 4" 

Centered Formulas for Numerical Differentiation. Upon differen- 
tiating the two Newton-Gauss formulas given in the preceding section it 
develops that for derivatives of even order every other term vanishes, 
and that for odd derivatives a similar result can be achieved by adding 
the two formulas and dividing by 2. This is Stirling's formula which, for 
a basic set of 2N + 1 points, is 

N x 

fc 2»-l /( 2»-l) (a . o) = £ ^.-[A 2 - 1 /- + A 2 *- 1 /!-*] + RN, 

s=m ■" 

R = m y (2iV+ l)l(2m - l)!c s+1 ,y +1 h2N +2m _ 2s _ l2N +2m _ 2s _ x 
N s = (2s + 1) \(2N + 2m - 2s - 1) ! 

For even derivatives over 2N + 1 points the result is 
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N 

h 2m f 2m \x Q ) = £ d m , s A 2s /_ s + R N , 

S—m 

p _ w v ( 2iV + l)!(2m)!c, +ltJN r + i (2iV+2w _ 2s) , (2iV+2m _ 2s) rt n 
RN ~ h(?s+lV.W + 2m-2s)\ ' {U ' 

Table 2 gives values of the c's and d's for derivatives of orders one through 
six for interpolations over as many as eleven points. 

Table 2. Coefficients for Stirling's Formula 



3 


Clj 


°2j 


1 


1 




2 


i 

6 


1 


3 


1 
30 


1 

T 


4 


1 
140 


7 
120 


5 


1 
630 


41 
3~02T 


6 


1 . 

27 720 


4 79 


151200 



JL 

3 
13 

TTT 

139 
"6048 



dij 


d 2 j 


dzj 


1 






i 

12 


1 




1 
90 


1 
6 


1 


1 
560 


7 
24 


1 

T 


1 


4 1 


1 3 


3150 


7560 


240 



Example. Find/' (3.4), /"(3.4) by Stirling's formulas by using the data 

for the example in the subsection on the Newton Interpolating Formulas. 

Here n . 

x = 3.4, 

hf'M = CuiCAf-i + A/ ) + c li2 |(A 3 /_ 2 + A 3 /_!), 

0.2/'(x ) = i(0.0263289 + 0.0248236) - ^(0.0001945 + 0.0001628) 

= 0.1277282, 

h 2 f"M = di.i A 2 /_! + d h2 A 4 /_2 

= -0.0015053 - tr(- 0.00003 17) 

= -0.03757. 

Since f'(x) = In e/x, f"(x) = In e/x 2 , the true values are 0.1277337 and 
0.0375687, respectively. 

Remarks on Numerical Differentiation. The formulas given above 
apply only to calculation of derivatives at tabular points. For derivatives 
at other points and for derivatives of functions tabulated at unequal 
intervals, it is possible, but not practical, to differentiate Lagrange's 
formula. The most obvious thing to do in this case is to form a new table 
at equal intervals which includes the point at which a derivative is re- 
quired. In the first case, the coefficients to be used in the interpolation 
formulas do not change in moving from one point to the next, which 
simplifies matters considerably. It is to be observed that the situation 
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then becomes complex as regards the remainder term. On the other 
hand, it is also to be observed that derivatives cannot be computed with 
an accuracy greater than the number of significant figures in the highest 
order difference used. If, therefore, an interpolation can be performed 
with sufficient accuracy to guarantee the last significant figure written 
down, differentiation on the interpolated table will give as much accuracy 
as is available from the original table. Similar remarks apply to the dif- 
ferentiation of functions having more than one variable. 

Integration 

General Remarks on Numerical Integration. The most useful 
characteristic of the integral, from the point of view of numerical integra- 
tion is the fact that 



r*h nC pb 

I f(x) dx = I f{x) dx + I f(x) dx, 

J a Ja *) c 



which means that the problem can be reduced to finding integrals over 
small ranges. 

There are many possibilities in developing formulas with regard to choice 
of basic sets of points for the interpolation polynomials to be used. The 
two sets commonly used will be discussed here: (1) equally spaced points 
starting with a and ending with b, which leads to a set of formulas called 
Cotes' s formulas; (2) a spacing developed by Gauss. 

Cotes's Formulas for Numerical Integration. Let 

xi = a + ih, i = 0, 1, • • •, n, 

with x n = b, that is, define h = (b — a)/ri. Let fi = f(x{). Then the 
formulas are as shown in Table 3. The formulas for n = 1 and n = 2 
are known respectively as the trapezoidal rule and Simpson's rule. Except 

Table 3. Cotes's Formulas 



r° h° 

§f(x)dx = \{U+h)--f«\£), 



n = 1 



^ (/0 + 4 , I+/2) _^|), „. a 

= ^ [7(/o + /.) + 32a! + / 3 ) + 12/,] - 8 *'{''' (t) . n = 4 
45 945 

= i^o [41(/o +/ 6 ) + 216(/i +/s) + 27(/ 2 +/0 

+ 272/ 3 ] - -~/ (8) 0l), n = 6 
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for n = 1, formulas for odd n are not given, since their remainder term has 
the same order of magnitude in h as the preceding even n. 

The choice of n is a matter of judgment. Generally, the larger n the 
less evaluation of integrand is required for given accuracy. If the inte- 
grand requires much computing, this is important. 

Example. If f(x) = e~ x /x, a = 1, b > a then \f n) (x)\ ^ 1.36n! If it 
is required that the remainder term shall not exceed 10~ 10 , then for the 
above n's the h's are 0.0008, 0.013, 0.03, 0.06, and the number of evaluations 
of integrand per unit b — a is 1250, 77, 33, 16 respectively. 

Gauss's Formula. For any n let 





Xi = a + (b — a)£i, 


» = 0, 1, ■•■An/2], 




= b — (b — a)&, 


i= [n/2] + l, •-., 


Then, for n 


= 2N 





X 



JV-1 



f{x) dx = (b - a)[A N S N + 2 MSi + S2N-i)b 

i=0 



for n = 2N + 1 






N 



S(x) dx = (6 
where the A's and the £'s are given in Table 4. 



a) 2 AiiSi +'S2N-i), 

i=0 





Table 


4. Values of . 


A i AND Xi 


in Gauss' 


s Formula 


n = 


1 


& 


= 0.21132 


48654 


A = 


0.5 




n = 


2 


£o 


= 0.11270 


16654 


A = 


5 
1 8 








fi 


= 0.5 




Ai = 


4 
9 




n = 


3 


So 


= 0.06943 


18442 


^o- 


0.17392 


74226 






Si 


= 0.33000 


94782 


A,= 


0.32607 


25774 


n = 


4 


So 


= 0.04691 


00770 


A = 


0.11846 


34425 






Si 


= 0.23076 


53449 


A,= 


0.23931 


43352 






s 2 


= 0.5 




A 2 = 


0.28444 


44444 


n = 


5 


So 


= 0.03376 


52429 


A = 


0.08566 


22462 






Si 


= 0.16939 


53068 


A,= 


0.18038 


07865 






s 2 


= 0.38069 


04070 


A 2 = 


0.23395 


69672 


n = 


6 


So 


= 0.02544 


60438 


A Q = 


0.06474 


24831 






Si 


= 0.12923 


44072 


A 1 = 


0.13985 


26957 






s 2 


= 0.29707 


74243 


A,= 


0.19091 


50253 






S3 


= 0.5 




A 3 = 


0.20897 


95918 



The remainder term is of order h 2n+1 . 
Hobson (Ref. 6). 



For further development, see 
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Other Integration Methods. Tchebysheff has developed a method in 
which the numerical integral has the form 

fctfo +/!+••• + /») 

which is useful if / represents data subject to uniform errors, since no error 
is weighted more than another. 

For multiple integration the methods given here may be applied re- 
peatedly. If the number of repeated integrations is quite large, the Monte 
Carlo method is useful. 

For integrals over an infinite range and for infinite integrands, trans- 
formations of the variable of integration can frequently be found which 
remove the difficulty. 



2. MATRIX INVERSION AND SIMULTANEOUS 

LINEAR EQUATIONS Murray Mannos 

General Remarks. The development of large scale electronic digital 
computers has made it numerically possible to invert many large size 
matrices and to solve large systems of linear equations heretofore con- 
sidered impractical because of their large size. Problems being attacked 
by matrix inversion include: 

(a) The numerical solution of a differential equation, a partial dif- 
ferential equation, or an integral equation satisfying boundary conditions 
is often achieved by resolving the problem into a large approximating set 
of algebraic equations. 

(b) A nonlinear problem is frequently replaced by a sequence of linear 
systems yielding successively improved approximations to the original 
problem. 

(c) Large systems of linear equations, at least in part, are serving as 
preliminary models for economic and business type problems. The object in 
linear programming (see Chap. 15), for example, is to maximize (minimize) 
a linear objective function such as profit (cost) subject to the restraints 
imposed by a system of linear equations (or inequalities). If the inverse 
of the matrix of coefficients of a linear system of equations is already 
known, the solution to the system is obtained by merely multiplying the 
inverse by the column vector whose components consist of constants on 
the right-hand side of the equalities. In the revised simplex technique 
(Ref. 7) designed for solving linear programming problems it is the inverse 
of certain basic column vectors that is calculated at each iteration or 
stage of the algorithm. 
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Practical ways of solving systems of linear equations are divided into 
two categories: the direct and indirect methods. 

(a) The direct method yields an exact solution in a finite number of 
steps provided no roundoff errors are permitted. 

(6) The indirect method usually involves an infinite number of iterations 
to get an exact solution. In practice one accepts the fact that one cannot 
get a precise answer but must be satisfied with a result sufficiently close 
to the exact result. At this point in the indirect method the calculation 
is broken off. To be really sure that the answer is sufficiently close either 
some estimate of roundoff errors must be made or the closeness must be 
determined perhaps by some physical considerations. Severity of roundoff 
errors may easily render useless results. 

The discussion will be confined to matrices whose elements are real 
and to linear systems whose coefficients are real. Many of the methods 
and results described apply equally well to the complex elements and 
coefficients simply by making appropriate word changes. Furthermore, 
any matrix of order n with complex coefficients may be represented by a real 
matrix of order 2n. 

No "best method" for either inverting matrices or solving linear systems 
of equations can be recommended. For a given technique, a matrix or 
a linear system of equations can always be constructed which will not 
work too well but which may work better with some other technique. 
In some cases it is a combination of methods, perhaps a direct followed 
by an indirect method, that works well for a system of linear equations. 
Ill-conditioned matrices, of which the favorite seems to be the Hilbert 
matrix, impose an extremely stringent test upon the accuracy of any given 
matrix inversion technique. A measure of the ill-conditioning of a matrix 
may be looked upon as the relative smallriess of its determinant compared 
with that of its individual elements. This will suffice here although more 
sophisticated measures could be used to interpret the notion of ill-con- 
ditioned matrices. The Hilbert matrix is denoted by H = (hif) where 
ha = l/i + 3 + 1 ih 3 = 1, 2, • • •, n). 

Having obtained by a given technique a not entirely satisfactory ap- 
proximation for the inverse of a matrix or for a solution to a system of 
linear equations, one may consider using techniques for improving the 
inverse of the matrix or the solution to the linear system of equations as 
the case may be. 

To facilitate the evaluation of procedures for matrix inversion or solution 
of linear systems for use on digital computers, a summary table of approximate 
storage requirements and number of operations is presented at the end of the 
section. 
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Matrix Inversion 

Each nonsingular square matrix A of order n has an inverse A ~ l such that 

(1) AA~ l = A~ l A = I. 

If for A = (aij) the elements a^ (i,j=\,---,n) are real, then the elements 
bij of A -1 = (bij) (i, j = 1, •••, n) are also real. If the a t y of the 
matrix A are specified, the problem is to find the numbers b{j of A~ l . 
For certain types of matrices this is relatively simple. 

(a) If D = (e^y) is a diagonal matrix, that is, d# = 0, i ^ j* and da ?± 
(i = 1, 2, • • •, n), then the elements of its inverse D~ l = (bij) are &;_,- = 0, 
i 7^ j, and 6« = 1/da (i = 1, 2, • • •, n). 

(b) If T = (a^) is a nonsingular lower triangular matrix, that is, aij = 0, 
i < j, and a^- ?± (i = I, 2, • • • , n), the elements of its inverse 2 1-1 = 5 
= (bij) can be obtained essentially by solving a series of linear equations 
in one unknown. Multiplying each of the columns of B by the first row 
of T yields 

on&n = 1; a nhj = (j = 2, • • •, n). 

This yields b n = l/«ii and b^ = (j = 2, • • •, n). Similarly, multiply- 
ing B by the second row of T gives 

«21&12 + G22&22 = lj «21&iy + 022% =0 (j = 1, • • • , U\ j ^ 2). 

Substituting the known bij (j = 1, • • •, n) into the latter equations yields 
new values b 2 j (j = 2, • • •, n) from the resulting n linear equations in 
each of these unknowns. By continuing in this way, multiplication of 
each of the columns of B by the nth row of T gives 

0>n\y\n ~T~ ftn2^2n T" " ' * "T" & nn b nn = lj 

anibij + a n2 b 2 j -{ h a nn b nj = (j = 1, • • •, n — 1). 

Substituting the known bij (i = 1, • • •, n — 1; j = 1, • • •, n) yields the 
values b n j (j = 1, • • • , n) of the last row of B. 

(c) An old standard method for inverting matrices is given by A~ l = 
(1/det A)(- - •), where the expression in parenthesis is the transpose of 
the matrix of cof actors of the elements a# of the given matrix A. This 
method is not to be recommended as practical for n greater than 3 or 4. 

(d) If one has already computed the characteristic polynomial or better 
still the minimum polynomial of a matrix 

m(x) = x m + aix m ~ x -\ h a m _ x x + a m ; a m ^ 0, 

then A -1 = (- l/a m )(A m ~ 1 -f- a x A m ~ 2 -\ \- a m _!.Z") since A satisfies its 
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minimum equation. In general it may be as much trouble calculating the 
characteristic or minimum polynomial as it is to invert the matrix itself. 

(e) Let Ai denote the ith. row of the nonsingular matrix A and 7; the 
^th row of the identity matrix. Then 

n 

Ai = J^aijlj (j = 1, • • -,'n). 
y=i 

If one has solved for the 7/s in terms of the A/s, then 



Ij — z2 bjkAk, 

and the matrix of coefficients of the latter equation is the desired inverse, 
i.e., A~ 1 = (bij). In general, this method is more cumbersome than a 
number of the methods described below. 

Jordan-Gauss Method. Write the matrix A with the identity matrix 
beside it as shown 



(2) 



A series of elementary row operations will be applied to A and these 
will also be applied in the same order to 7. When A has been reduced to 
7 by a series of elementary row transformations, then 7 will in turn be 
transformed into A -1 by the same transformations, and the process will 
be finished. If A is nonsingular, then for some i — 1, • • •, n it follows 
that an 7^ 0. One can by an exchange of rows guarantee that the element 
in the first row of the first column is different from zero. 

In case the matrix (an) has been altered by an exchange of rows one now 
denotes the left-hand matrix of (2) by (bij). Then adding to the ith row 
— bn/b n times the first row (i = 2, - • • , n) the new left-hand matrix of 
(2) takes the form 



~a n 


«12 ' 


«ln 


1 


• 


•• 


«21 


a 22 


O'ln 





1 • 


•• 


-«nl 


&n2 


&nn 





• 


• 1 



(3) 



b n c 12 
c 22 



C2n 



Lo 



Cn2 



The minor of order n — 1 in the lower right-hand corner of the matrix 
(3) has rank n — 1 so that at least one of the elements c 2 y 9^ (j = 2, • • • , 
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n). Applying the same argument as before to this minor, all elements below 
the diagonal element of column 2 of the left-hand matrix in (2) may be 
reduced to zero. Similarly the element in the first row, second column 
may be reduced to zero. The first column remains unchanged while the 
second one has been altered to the desired form. 

By continuing in this way the left-hand side of (2) may be reduced to 
the diagonal form 

'bii • • • 



d 



22 



Lo o 



o 



"nn- 



with each diagonal element being different from 0. By dividing the first 
row of (2) by b n , the second row by d 22 , etc., the left-hand matrix of (2) 
is finally reduced to the identity and the right-hand matrix is now A~ l . 

The diagonal elements b n , d 2 2- • ■ of the first, second, • • • columns which 
are used to reduce the remaining elements of their respective columns to 
zero are referred to as pivots. Care should be exercised whenever possible 
not to select a pivot which is too small or too large; otherwise, loss of sig- 
nificance among other difficulties may arise. Numerous variations of the 
use of elementary row operations for inverting matrices exist in the litera- 
ture (Ref. 8). 

Partition Method. Let the nonsingular n Xn matrix A be partitioned 
as 

'An A l2 

.A 2 i A 2 2- 



A = 



where A n is an m X m minor (m < n) which is likewise nonsingular. 
Then the inverse A~ 1 of A is given by the matrix 



A~ l = 


--B21 


B12 
B22 


where B n = An' 1 + XA _1 F, 






B 12 = -XA- 1 , 






B 2l = -A-'Y, 






B 2 2 = A , 







and 



X = An-'Ati, Y = A 2l A n -\ 

A = A 22 - YA 12 = A 22 - A 21 X. 
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Inverting a matrix of order n has been reduced to inverting a matrix of 
m, and another of order n — m. However, one has to pay the price of 
performing a number of matrix multiplications afterwards. 

Morris Escalator Method. By starting with the inverse of the 
2X2 principal minor M 22 in the upper left-hand corner of the nonsingular 
matrix A one may by the partition method obtain the inverse of the 3X3 
principal minor Af 33 in the upper left-hand corner of A. Then ikf 33 is 
used to compute the inverse Af 44 of the 4X4 principal minor in the first 
four rows and columns. Step by step, one dimension at a time, the parti- 
tion procedure is carried out until A -1 is obtained. The process is un- 
interrupted until the inverse of one of the Ma fails to exist, a fact which 
is established by noting that the corresponding A { = 0. This situation 
is remedied by interchanging the ith row with an appropriate row, say the 
jth, of the remaining n — i rows of A, computing the inverse of the new 
i X i principal minor in the left-hand corner, and then continuing as before. 

In order to obtain A -1 one must interchange the it\i andyth columns of 
the resulting inverse so obtained. If several of the inverses of principal 
minors encountered fail to exist, a similar procedure applies in each instance. 

Gram Schmidt Orthogonalization Method. Premultiplication of 
the nonsingular matrix A by an appropriate matrix P transforms A into an 
orthogonal matrix, i.e., 



(4) 



PA = 0. 



Since the inverse of an orthogonal matrix is its own transpose, it follows 
from eq. (4) that 

A- 1 = A'P'P, 



where P = DN 



N = P 



n— 1 



P 2 P 1 ;P i _ 1 = 



1 
1 











• 


•• 1 





Ci\ 


C»2 * 


C{,i—1 


1 





• 


•• 


1 











1J 
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D = 



and in turn 



"i/IQi. 
o 





1/IQ2I 



i/IQ»|J 



Cj"i 



A iQ j 



j= 1,2, •••,*-!, 



where A{ denotes the ith. row of A, Qj denotes the jth row of Q = NA, and 
\Qi\ denotes the length of the ith row of Q considered as a row vector. 

Inversion of Modified Matrices. If the inverse of a matrix A is 
known, the inverse of a matrix differing from A in only an element, a row, 
or a column can be found as a result. If the matrix differs from A by several 
elements, rows, or columns, its inverse may be realized by repeated ap- 
plication of this method. The method is based on the matrix identity 



(5) 



(A + xy')- 1 = A' 1 - 



(1 + y'A-'x) ' 



where x and y are arbitrary column vectors. The matrix xy' can be made 
to consist of all zeros except the element in the ith row and jth column 
where it is to contain a fixed value c. This is easily achieved by taking 
x = cei and y = e 3 - where e* is the unit column vector containing a 1 in 
the ith position and elsewhere. By taking y = a the matrix xy' has x 
for its ith column and all other columns consist of zeros. Hence, if the 
vector x stands for the vector difference of the ith column of the matrix 
whose inverse is desired and the ith column of A, the required inverse is 
obtained from eq. (5). A similar argument applies if the matrices differ 
only in one row. 

Improving a Computed Inverse (Hotelling and Bodewig, see Refs. 
9 and 10). Suppose that the matrix C is considered a sufficiently good 
approximation to the inverse of the matrix A so that B — I — AC has 
very small elements. If necessary, for some specific purpose, the computed 
inverse can be improved by forming the sequence 



C k = C*_!(/ + B 2 "-), 



k = 1, 2, 



Actually, the sequence converges to A J and so A J is expressible in the 
following form of an infinite product 
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(6) A-'^CoIKl + B 2 "- 1 ). 

fc=i 

Very frequently the improvement found by computing C (I + E) or 
perhaps C (I + B)(I + B 2 ) is sufficiently satisfactory. Although there 
are a number of variations other than eq. (6) for expressing A -1 , the 
present scheme has some merit when using an electronic digital computing 
machine, since it is only necessary to keep successively squared powers of 
B, adding this to the identity matrix /, and premultiplying by the last 
computed approximation to A -1 . 

Systems of Linear Equations. Direct Methods 

Direct methods arrive at an exact solution in a finite sequence of arith- 
metical operations. 

Elimination. Given a set of m ^ n linear equations in n unknowns 

a n Xi + a 12 x 2 -1 1- ai n x n = &i 

a 2 \x x + a 22 x 2 -\ \- a 2n x n = b 2 



or more briefly in matrix notation 

Ax = b, 

the augmented matrix (A | b) is operated on by a sequence of elementary 
row operations which reduce the matrix of coefficients A to echelon form 
(see Chap. 3). If a row of the reduced form of (A\b) is of the form (0, 
0, • • •, 0, c), where c^O, the system (7) is inconsistent; otherwise, it is 
consistent. Arbitrary values are assigned to those x's which do not cor- 
respond to a leading coefficient of 1 in some line; while the remaining x's 
may be solved for in terms of these parameters one at a time as a linear 
equation in one unknown whose coefficient is 1. 

Note. In the remainder of this section only the case with m = n and 
the matrix A nonsingular will be considered. 

Use of Cramer's Rule. Let A (fc) denote the matrix constructed from A 
in (7) by replacing column k by the column b of right-hand coefficients. 
Then the unique solution to (7) is given by Cramer's rule in the following 
form as a ratio of determinants 

detA(fc) 

x k = —-—- ( fc = !> •••> n )- 
detA 

For n > 3 or 4 this method is not to be recommended as efficient. 



NUMERICAL ANALYSIS 14-21 

Known Inverse. If the inverse A~ l of A has already been calculated 
by any of the previously described or perhaps other methods, the solution 
in matrix form is given by 

x = A~ x b. 

However, if A~* must be computed for the sole purpose of getting x, the 
method is not always efficient for large values of n. 

Conjugate Gradient Method. Most of the iterative schemes involve an 
infinite number of iterations and so are classified as indirect methods. 
However, an outstanding iterative scheme called the conjugate gradient 
method involves but a finite number of iterations and so is classified as 
a direct method. Because of the way in which the algorithm for this 
scheme is built up, it seems more appropriate to discuss it after the gradient 
method, an indirect method. The elegant finite algorithm for the con- 
jugate gradient method seems to have been independently discovered by 
Stiefel, Hestenes, and Lanczos (Ref. 11). For a linear system Ax = b, 
det A 9^ 0, of n equations the algorithm starts with an initial guess x 
building up successive approximations x\, ■ ■ ■ , x n and finally terminates 
after at most n of these steps or iterations. The corresponding residual 
vectors 

ri = Axi — b (i = 0, 1, • • •, n) 

so formed are mutually orthogonal to the preceding ones. If r; ^ 
(*" = 0, 1, • ■ •, n — 1) then r n orthogonal to each r,- means r n must be the 
null vector 0; since n -\- \ linearly independent vectors of dimension n 
cannot exist. 

Systems of Linear Equations : Indirect Methods 

By and large this discussion includes most iterative methods since it 
takes an infinite number of steps to carry through the whole process. An 
iteration for solving a system of linear equations is a set of rules for operat- 
ing on an approximate solution (xi (fc) , • • •, x n (fc) ) to obtain an improved 
or more precise solution {x^ k+l \ •••, x n ^ k+1) ). The sequence of ap- 
proximate solutions so defined must converge to the actual solution of the 
given system of equations. In some cases it is a pronounced advantage 
to start out with a rather good initial approximation (xi (0) , • • •, # n (0) ), 
whereas in others this is not necessarily true. It is frequently advantageous 
to improve the solution obtained by a direct method by a few iterations, 
since the direct solution usually is afflicted with roundoff errors. 

Seidel Method. One starts off with a guess (xi (0) , x 2 m , •••, x n (0) ) 
as the initial solution to the linear system (7). Substituting in the first 
equation of (7) the values a; 2 (0) for x 2 , rK 3 (0) for x 3 , • • •, and finally a: n (0) 
for x n , and then solving for x yields a new value £i (1) as the first component 
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of the next approximate solution. Next by substituting in the second 
equation of (7) the newly gained value Xi (1) for xi and x 3 (0) for x 3 , • • •, 
a; n (0) for x n , and then solving for x 2 , one obtains a new value x 2 (1) as the 
second component of the next approximation. Continuing in this fashion 
and finally substituting in the nth or last equation of (7) the values x^ for 
x\, x 2 (1) for x 2 , • • • , £n-i (1) for x n _i and solving for x n yields the final 
component x n w of the new iteration (#i (1) , £ 2 (1) , •••, £ n (1) ). The ap- 
proximation (#i (1 \ x 2 ^\ • • •, # n (1) ) is used primarily in the next iteration 
to obtain the improved approximation (xi (2) , £ 2 (2) , •••, £ n (2) )- One 
continues in this way. 

The process is very well adapted to machine usage. Convergence is 
assured when either the matrix of coefficients A is positive definite or when 
the diagonal element of the ith row dominates the rest of the row for eacli 
i, that is, when 

\a»\ > 2 K;l 0' = 1, 2, • • •, n). 

Convergence is also guaranteed for additional types of matrices, and there 
are a number of variations of this procedure. In particular, the "back 
and forth" Seidel method due to Aitken and Rosser was especially de- 
signed to handle those cases in which convergence of the regular Seidel 
method was erratic. 

Relaxation Method. First write the system (7) in the form 

&1 — CL U X\ — ai 2 X 2 CLlnXn = 

b 2 — a 2 iX\ — a 22 x 2 a 2n x n = 

(8) 



o n — «ni^i — a n2 x 2 a nn x n = 

and assume that none of the diagonal elements an (i = 1, • • •, n) is equal 
to zero. Then take x (0) = (zi (0) , rc 2 (0) , • • •, x/ 0) , • • •, x n (0) ) as an initial 
guess to the solution. If it should accidentally happen that x (0) satisfies 
(8), one is finished. If not, define the residual vector by r (0) = (ri i0) , 
r 2 (0) , • • •, r n (0) ), where r/ 0) (i = 1, 2, • • •, n) is the value or residual ob- 
tained by substituting a; (0) in the left-hand side of the ith equation of (8). 
Suppose that r/^ is a component of largest magnitude in r (0) . The object 
then is to reduce the residual r; (0) to by altering the value of the ith 
component x/ 0) of x (0) while keeping the remaining components of x (0) 
fixed. The next trial solution x (1) is constructed as follows: 

x k (1) =x k {0) (fc= 1, •••,n;fc?*i) 

r .(0) 

T .d) _ T .(0) i '2 

a{{ 

This effects a new set of residuals r (1) with ith residual equal to 0. Select 
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the residual of maximum magnitude and similarly apply the same scheme 
as above to obtain £ (2) . This process is repeated again and again so as 
ultimately to reduce all residuals to as close to as possible. 

It is sometimes possible to speed up convergence by picking residuals 
not necessarily of maximum magnitude. In fact, by varying several of 
the variables at one time it may be possible to speed up convergence con- 
siderably. However, it would be very difficult to write a code including 
many such variations and tricks. 

Note. In the following sections it is often convenient to introduce a 
measure or metric different from the usual one in order to cut down on the 
amount of computation required. 

Approximations. Let A be a symmetric positive definite matrix, then 
the length of a vector x with respect to the metric A is defined as \x\a = 
(x'Ax)^, and any two vectors x and y are conjugate or A-orthogonal if 
x'Ay = 0. These are extensions of the usual definitions of length and 
orthogonality. The latter may be obtained from the new definitions by 
taking A = I. 

With respect to the usual metric, | Ax — b\ 2 = if and only if Ax — b = 
0. This means that solving Ax = b is equivalent to finding an x such that 
\Ax — b\ 2 is minimized, since it is known that is its minimum value. 
Likewise with respect to the generalized metric B, \Ax — b\s 2 = if 
and only if Ax — b = since B must be positive definite. 

Now let 

(9) f(x) = \Ax-b\ B 2 
and consider the family of hyperellipsoids 

(10) f(x) = k, 

where k may take on any constant value. Then the solution of the system 
Ax = b is the common center of the family of ellipsoids eq. (10). The 
game then is to construct a set of approximations rc (0) , x (1) , • • • which get 
us to or close to this center. The more rapidly this happens the less 
computation is involved. 

Gradient Method. Start with a guess x w as an initial approximation 
to the solution of Ax = b. The ellipsoid of the family (10) obtained by 
setting k = /(x (0) ) passes through the point z (0) in n-dimensional space. 
Then proceed in the direction of the gradient of —f(x) at z (0) that is, 
along the inner normal to the ellipsoid f(x) = /(x (0) ). It is known that 
f{x) decreases most rapidly along the latter direction and so it is natural 
to proceed in this direction until one arrives at the minimum of f(x) along 
this inner normal. This happens at that point x w where the inner normal 
becomes a tangent to one of the family of ellipsoids in eq. (10). Similarly, 
proceed along the inner normal of the ellipsoid f(x) = /(a; (1) ) until the 
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minimum of f{x) in this direction is reached. Continue in this way and 
work in closer and closer to the common center of the family of ellipsoids 
in (10). 

The algebraic procedure for solving Ax = b with respect to the metric 
B according to the geometric scheme described above is as follows: 

(a) Compute C = A'BA 

c = A'Bb. 

(b) Make an initial guess x (0) . 

(c) Use the following algorithm to obtain the approximation £ (l+1) 
from that of x (l) . (i) Calculate the vector z M in the direction of the gradient 
of f(x) at x w ; i.e., z (i) = Cx w - c. (ii) Calculate 



«i = 



(z (i) )'z w 
( z d)yCz w 



(iii) Obtain x (l+1) = x (l) — a t -2 (,) , where the coefficient a* determines the 
minimum value of f(x) in eq. (9) along the inner normal to f(x) = f(x M ) 

If A is a symmetric positive definite matrix, it is most convenient to 
choose the metric B = A -1 , for then A replaces B and b replaces c through- 
out the above algorithm with a resulting simplification. 

A considerable advantage of the gradient method is that there need 
not be an accumulation of roundoff error since the vector g w along the 
gradient can be recalculated for each iteration. The function f(x) in eq. 
(9) may be regarded as a measure of the closeness of an approximation 
x to the true solution A~ l b. For the gradient method it is true that 
f{x {l+l) ) < f(x M ) for each i and that /(# (l) ) approaches in the limit; 
that is, x M converges steadily toward the true solution A~ x b. However, 
it is still true that the convergence may be slow or, in other words, it may 
take many iterations to get close to the center of the ellipsoids. A number 
of variations of the gradient method have been devised to try to speed up 
the convergence. 

Conjugate Gradient Method. First consider the case where A is 
symmetric and positive definite. The object in the conjugate gradient 
method as in the gradient method is to get to the common center of the 
family of ellipsoids eq. (10). However, the route taken in the conjugate 
gradient method is different from that of the gradient method and is so 
modified as to get to the center of the family eq. (10) in but a finite num- 
ber of steps, namely, at most n iterations. 

The procedure in three dimensions will be described. The discussion in 
higher dimensions follows along similar lines. 

As before, make an initial guess z (0) and proceed from rc (0) along the 
negative gradient of f(x) or what is the same along the inner normal of 
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the three-dimensional ellipsoid f(x) = /(a; (0) ). Take as the next ap- 
proximation the point z (1) , which is the midpoint of the resulting chord of 
the ellipsoid f(x) = /(x (0) ). Consider the diametral plane through £ (1) 
containing the locus of midpoints of the chords of f(x) = /(£ (0) ), which 
are parallel to the direction of the inner normal. The diametral plane so 
formed cuts out a two-dimensional elliptic cross section from the ellipsoid 
f(x) = /(a: (0) ). The common center of the ellipsoids (10) of interest lie 
in this two-dimensional elliptic cross section, and the method is designed 
so that all subsequent approximating points shall remain trapped in this 
cross section. The diametral plane of f(x) = /(x (0) ) is likewise a diametral 
plane of the interior ellipsoid f(x) = /(z (1) ) of the family (10) and cuts 
it in a two-dimensional elliptic cross section lying within the previous one 
cut from f(x) = f(x w ). Next proceed from x (1) along the gradient of 
f(x) within the last elliptic cross section formed and take for x (2) the 
midpoint of the chord so formed in the ellipse. In other words, instead of 
proceeding from x (1) along the inner normal of the ellipsoid f(x) = /(x (1) ) 
as in the gradient method, proceed along the inner normal of its cross 
section made by the diametral plane through a; (1) . 

Again the locus of centers of chords parallel to the chord through £ (1) 
and :c (2) forms a diameter of the elliptic cross section of f(x) = /(x (1) ), 
which contains not only x (2) but also the center of the family (10). Next 
proceed from rc (2) along this diameter, choosing its center as the new and 
final approximation x (3) . By barring roundoff error, £ (3) yields the exact 
solution to a linear system of three equations in three unknowns. If either 
of the chords mentioned above passing through £ (0) , x (1) happens to pass 
also through the center of the family (10), the process will end in only one 
or two iterations, respectively, instead of three. This will be indicated by 
the residual r (1) = or r (2) = 0, respectively. 

Algorithms for a symmetric positive definite matrix of order n and for the 
general n-dimensional case, respectively, will be given below, where pi 
denotes a vector in the direction of x (t) to x^ l+1 \ 

(a) Pick x w ; then let p (0) = r (0) = b - Ax {0 \ 

| r (i)|2 

(&) a* = 



( v wyApW 
( C ) x (i+1) = x {i) + a iV {i \ 
(d) r (i+1) = r (i) - a { Ap {i \ 



(i+l) 12 



(e) h = 



(/) p(*+D = r^+D + bip™, 
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where the coefficient a t - is selected to make # (l+1) the appropriate distance 
from x (i) and b{ to keep p (t+1) in the appropriate direction as described 
above. 

The algorithm eqs. (11) may be applied to a matrix which is symmetric 
and positive semidefinite as well as to a symmetric positive definite matrix. 

In the case where A is a general matrix, the system Ax = b is replaced 
by the equivalent system 

(12) A' Ax = A'b, 

where A' A is a symmetric and positive semidefinite. The algorithm 
(11) could thus be applied to eq. (12), but in order to avoid the roundoff 
errors due to computing A' A, it is better to use the following algorithm 
which leads to theoretically equivalent results. 

(a) Pick xS 0) , then let r (0) = b - Ax w , p (0) = A'r m , 

| AV «|2 



w 


Wi — 


|Ap<*>| 2 




(c) 


x (i+l) 


= x (i) + a 


iV {i \ 


(d) 


r (i+D 


= r (i) - a, 


A V «\ 


(e) 


*-i 


^_/ r (i+l) 12 




|AV (<) | 2 




(/) 


p(*+D 


_ j±> r (i+l) 


+ 6,-p w . 



The conjugate gradient method has numerous advantages in addition 
to those already mentioned. One may start all over again with the last 
approximation obtained as the initial approximation in order to nullify 
the effects of accumulated roundoff errors. Also, each successive ap- 
proximation is better than its predecessor. It is very important to note 
that the given matrix is unchanged during the procedure so that the origi- 
nal data are used again and again. This permits use of special properties 
of the given matrix such as its particular form or sparseness. A number 
of variations of this technique have been devised. 

A great many of the most important works in the field are to be found 
in the extensive bibliographies of works by Forsythe and Householder 
(Refs. 8, 9, 12, and 13). 

Computer Storage Requirements and Number of Operations. 
Storage requirements for a given problem will vary in general with the 
machine, with the programmer, and with the layout of the program. 
Hence, in Table 5 the number of storage locations required for the program 
of a given technique of matrix inversion or solution of a linear system shall 
simply be denoted by the symbol w. 
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A multiplication or a division will be identified simply as a multiplication. 
Likewise an addition or a subtraction will be identified as an addition. 
Since a multiplication requires from about 2 to 10 times as much time as 
an addition on most computers, greater weight should be accordingly 
apportioned to the number of multiplications. If the number of multiplica- 
tions required for a given technique turns out to be, for example 2n 3 -f- 3n -f- 
1 ; then 3n + 1 is negligible compared with 2n 3 when n is sufficiently large. 
One says the number of multiplications required in this case is of the order 
2n 3 , and this is simply indicated by 2n 3 . 

In the case of the indirect procedures such as the Seidel, relaxation, and 
gradient methods the number of iterations necessary for a satisfactory 
solution varies from problem to problem. In fact, the number of iterations 
required depends upon the original system of equations, the choice of the 
initial solution, and the accuracy stipulated beforehand. In these cases 
storage requirements and the number of operations are given for one itera- 
tion. For the conjugate gradient method these will be given totally for 
all n iterations. 



Table 5. Computer Storage Requirements and Number of Operations 
for Matrix Inversion and Linear Systems of Equations 

n = the order of matrix involved 

w = the number of storage locations required for the computer program 
of a given technique. 



Method 


Storage Requirements 

Matrix Inversion 


] 


Multiplications 


Additions 


Jordan-Gauss 


2ft 2 + w 




ft 3 


7l S 


Morris escalator 


ft 2 + w 




fft 3 


w 


Gram-Schmidt 


|-n 2 + w 




J/ft 3 


Jjf-ft 3 


Modified matrix 


?i 2 + 2n + w 


(a) 


one element ft 2 


(a) ?i 2 






(b) 


one row or 
column 2ft 2 


(b) 2ri 2 






(c) 


whole matrix 2ft 3 


(c) 2ft 3 




Linear Systems of Equations 




Elimination 


ft 2 + ft + w 




ft 3 /3 


ft 3 /3 


Seidel 










(one iteration) 


ft 2 + ft + w 




ft 2 


ft 2 


Relaxation 










(one iteration) 


ft 2 + n + w 




ft 2 


ft 2 


Gradient 










(one iteration) 


n 2 + 5n + 1 -+- w 




2ft 2 


2ft 2 


Conjugate gra- 


Symmetric positive definite 








dient 


2ft 2 + 6ft + 2 + w 




ft 2 


?i 2 


(one iteration) 


General case 










4ri 2 + 5ft + 2 + w 




3?? 2 


3ft 2 
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3. EIGENVALUES AND EIGENVECTORS Murray Mannos 

General Remarks. The characteristic equation of a matrix together 
with the corresponding eigenvalues (characteristic values) and eigenvectors 
(characteristic vectors) plays a fundamental role in the theory of mechanical 
or electrical vibrations. Examples: the nutter vibrations of an airplane 
wing, the elastic vibrations of a skyscraper or bridge, the buckling of an 
elastic structure, the transient oscillations of an electric network, and 
mechanical wave vibrations of molecules and atoms. Similar remarks 
concerning direct and indirect methods, roundoff errors, etc., apply to 
the finding of the eigenvalues and eigenvectors as to the inverting of a 
matrix and the solution of a linear system of equations (see Refs. 8, 9, 
and 12-14). 

In practice, it usually happens that all the eigenvalues of a matrix are 
distinct. This gives rise to a matrix A which can be diagonalized by a 
similarity transformation. Under a similarity transformation the eigen- 
values of A remain invariant. A symmetric matrix can be diagonalized 
by an orthogonal transformation and similarly a Hermitian matrix can 
be diagonalized by a unitary transformation. Hence, these types of mat- 
rices are frequently singled out for special treatment by somewhat less 
general methods than apply to the most general type of matrix. Matrices 
which cannot be diagonalized by means of a similarity transformation 
or whose eigenvalues are multiple or very closely spaced cause the proce- 
dures to become more complex. Results concerning the bounds of eigen- 
values are sometimes useful in helping to isolate them. In numerous cases 
it suffices to find either the dominant or the least eigenvalue. 

The elements of the matrix A will usually be complex elements but they 
may be confined to be real numbers in some instances. The matrix A 
itself will always be of finite order. 

Approximations for digital computer storage requirements and number of 
operations for finding the eigenvalues and eigenvectors of a matrix cannot 
be given as readily as in the cases of matrix inversion and the solution of 
systems of linear equations. This is because the solution of an eigen- 
value problem often consists of a number of major segments, such as an 
iteration, the reduction of a matrix to a direct sum of triple diagonal 
matrices whose sizes depend on the original matrix, the solution of complex 
equations, or the evaluation of transcendental functions at specific places, 
or the consideration of a sequence of Sturm functions. In the case of the 
triple diagonal method, consideration of the computer aspects has been 
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broken down in terms of the more important segments. Similarly, com- 
puter information for one step of the reduction process for finding eigen- 
values of a symmetric matrix by the Jacobi method is also given in Table 
6 at the end of this section. 

Characteristic Polynomial. The characteristic polynomial f(x) of a 
matrix A of order n over the complex number system may be defined as 



(13) det (XI - A) = det 



X — an — a 12 • • • — a ln , 

— a<L\ X — (222 ' • • ~^2n 



~&n\ Q>n2 ' ' ' A Q>nn- 

= X* + c 1 X w - 1 +..-+c n 
= /(X). 

The matrix XI — A has elements which are polynomials in X with com- 
plex coefficients. The characteristic polynomial may be found by the fol- 
lowing methods: 

1. The theory of determinants for such matrices is developed along the 
same line as for those matrices whose elements are real or complex numbers. 
Hence, the det (XI — A) can be expanded along any row or column to obtain 
its characteristic polynomial. This method is not to be recommended for 
n > 3. 

2. The coefficients C\, c 2 , • • •, c n of the characteristic polynomial in eq. 
(13) may be obtained from subdeterminants of the matrix A itself: c 1 = 
— (o-x x + a 2 2 + • • • + d nn ) is the negative of the sum of the diagonal ele- 
ments of A or simply the negative of the trace of A ; c 2 is the sum of the 
determinants of the 2X2 principal minors of A (i.e., the totality of minors 
having two of their elements on the diagonal of A); c 3 is the negative of 
the sum of the determinants of the 3X3 principal minors of A, • • •, c n 
= (— l) n det A. Likewise, this method is not to be recommended for 
n > 3. 

3. A finite iterative scheme based on repeated premultiplication by the 
matrix A yields the coefficients Ci, c 2 , • • •, c n of (13) also. This is the so- 
called Souriau-Frame algorithm. 



Ci = — trace A\, 



B X =A X + cj 



At =A, 

Ah = ABk-i, 

K 

(k = 2,S, ..., n ) 
4. Another way of finding the characteristic polynomial of a matrix A is 



c k = — trace — , B k = A k + c k I 

k 
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to build it up one degree at a time by finding the characteristic polynomial 
of the upper left-hand minors of A in increasing size. 

Let M{ denote the upper left-hand minor of A of order i, I { the unit 
matrix of order i, and fi(\) the characteristic polynomial of M{. 

Since 

(Xli - Mi) adj (Xli - Mi) = fi(\)I, 

it follows from a consideration of the last column that 



(14) 



where 



XL 



"MX) " 

&2i(A) 


= M t 


~b u (X) ■" 

MX) 


+ 


'0 " 



bt-i.t(X) 

-fi-M - 




Mi,;(X) 
-/;-i(X) . 






-A-(x). 



'MX) 

MX) 



Mi,i(x) 

L/i-i(A) J 

is the ith or last column of adj (Xli — Mi). 

From the first i — 1 rows of the expressions in eq. (14) the coefficients 
of the polynomials bki(X) (/c = 1, 2, ■ • •, i — 1) are determined by compar- 
ing the various powers of X. The leading coefficient of each of the 6/a(X) 
(k = 1, 2, • • •, t — 1) is determined by comparing coefficients of X* -1 . 
Then by using these known coefficients and by comparing coefficients of 
X t_2 , the second coefficients of each of the polynomials bki(X) (k = 1, 2, 
• • •, i — 1) are obtained. By continuing in this way the bk% (k = 1, 2, • • •, 
i — 1) are completely determined. If the known bui (k = 1, 2, •••,{— 1) 
are now substituted in the resulting equation formed by setting the ith or 
last rows of eq. (14) equal, the polynomial fi(\) is determined. 

One first forms /i(X) = X — an and uses the above technique to find 
/ 2 (X) from/! (A), etc., until finally /(A) = / n (A) is obtained from/ n _ 1 (A). 

5. The method of finite iterations may be used to obtain a polynomial 
equation from which some of the eigenvalues of a matrix A may be obtained. 
Let a; ^ be an arbitrary vector and form Ax. If x and Ax are linearly 
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independent, form A 2 x. If x, Ax, and A 2 x are linearly independent, form 
A' s x, etc. Continue in this way until one ultimately comes to a sequence x, 
Ax, A 2 x, •••, A k x, which is linearly dependent. This must happen for 
k ^ n, since at most n vectors are linearly independent. That is, 

(15) (A k + dA*" 1 + c 2 A k ~ 2 + • • • + c k I)x = 0. 
Form the corresponding polynomial 

(16) P t (X) = \ k + c^- 1 + c 2 \ k ~ 2 +•■•+(*. 

The polynomial P&(X) of eq. (16) is a factor of the minimum polynomial 
ra(X) of A, which will be defined explicitly in the subsection on eigenvalues 
and eigenvectors. If k = m where m is the degree of ra(X), then Pfc(X) 
coincides with the minimum polynomial ra(X). Finally if k = n, then 
Pk(X), the minimum polynomial m(X), and the characteristic polynomial 
/(X) all coincide. 

The coefficients c lf c 2 , • • •, c k are obtained from eq. (15) by forming a 
set of linear equations resulting from a comparison of components. 

6. The necessity of testing for linear dependence and for solving a sys- 
tem of linear equations are disadvantages of the method of finite iteration. 
However, the polynomial eq. (16) may be obtained while avoiding these 
disadvantages by the so-called method of minimized iterations due to 
Lanczos (Ref. 47). 

Lanczos employs a finite algorithm involving the sequences of poly- 
nomials: 

Po(X) = 1 

P X (X) = (X - ao)Po(X) 

P 2 (X) = (X - aOPi(X) - b Po(X) 



P,-(X) = (X - di^Pi^Ck) - & l _ 2 P l -_ 2 (X) 



and the vectors given by the equations: 

(17) aj,-_i = P i - l (A)x , Vi-i = Pi-i(A')y , 

where 

y'i-xAxi-x y'i-iXi-t 

(18) a;_i = — 6i_ 2 = — > 

y i-M-i y i-2Xi- 2 

and x ^ 0, yo 9^ are not orthogonal but otherwise arbitrary vectors. 
The algorithm proceeds to calculate the vectors £;_! and y i _ l until one 
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of them becomes zero and the process terminates. From x and y one gets 
a from the left-hand equation of (18) by setting i = 1. This determines 
the polynomial -Pi(X) and in turn one gets the vectors X\ and iji from eq. 
(17) by setting i = 2. From x\ and 2/1 one gets the coefficients a x and b 
from (18) by setting i = 2. This in turn determines P 2 (X) from which one 
determines the vectors x 2 and y 2 by the use of eq. (17) with i = 3. Con- 
tinuing in this manner ultimately shows that either the vector Xk = or 
yi c = for some k. When this occurs the polynomial Pk(X), whose coeffi- 
cients are now determined, is singled out. The polynomial Pk(X) as before 
is a factor of the minimum polynomial m(X) and coincides with m(X) if 
k = m, and with the characteristic polynomial /(X) if k = n. 

Determination of Eigenvalues and Eigenvectors. /(X) = is called 
the characteristic equation of the matrix A and the n roots of this equation 
are called the eigenvalues of the matrix A. From eq. (13) it follows that if 
X is an eigenvalue of A, then det (XI — A) = /(X) = so that the system 

of linear equations , 

Ax = Xx 

has a nontrivial solution x ^ 0, and any such solution x 5^ is called an 
eigenvector of the matrix A. 

Once the coefficients of the characteristic polynomial have been deter- 
mined, the characteristic equation can be solved by Graeffe's, Bernoulli's, 
or any other known method for solving a polynomial equation to obtain 
the eigenvalues of A. If n is fairly high, a large amount of precision in the 
calculations must be exercised or roundoff error may easily invalidate the 
results. 

Apart from multiplicity it is possible to find the eigenvalues of A by 
considering a polynomial equation of lower degree than n. In this connec- 
tion the minimum polynomial of the matrix A will be defined below. By 
the well-known Cayley-Hamilton theorem it follows that f(A) = 0. In 
general, however, A satisfies polynomial equations of lower degree than 
n = deg/(X). One denotes by m(X) that polynomial of lowest degree with 
leading coefficient 1 such that m(A) = 0. This polynomial is unique. 
Furthermore, the minimum polynomial w(X) divides the characteristic 
polynomial /(X), and each of the eigenvalues of A is a root of m(X) = 0. 
The multiplicity of such a root X of m(X) = is less than or equal to the 
multiplicity of X as a root of /(X) = 0. Hence, if a certain procedure leads 
to the construction of the minimum polynomial of A, it may be sufficient 
to obtain the necessary information concerning the eigenvalues of A from 
m(X), which may be of considerably lower degree than the characteristic 
polynomial of A and so easier to work with. If one denotes by g(X) the 
greatest common divisor of the polynomial elements of adj (XI — A) it 
may be shown that 
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m /(X) 
m(X) = 



Direct Methods 



Apart from roundoff errors, the procedures described under the heading 
of direct methods terminate in a finite number of steps with exact results. 

The Escalator Method. If the eigenvalues of a symmetric matrix 

A{ of order i are known and distinct and the eigenvectors are also known, 

the symmetric matrix 

\Ai a i+ i 
A i+ i = , 

La i + i a>i+i,i+i. 



obtained by bordering A{ with an additional row and column also has eigen- 
values and eigenvectors which can be found in terms of the eigenvalues and 
eigenvectors of A{. Furthermore, the eigenvalues of Ai +i are distinct and 
interlace with those of A{. 

Let Xfc (fc = 1, 2, • • •, i) denote the eigenvalues of Ai, and u k denote the 
eigenvectors of A{. Then the eigenvalues of A l+ i are obtained by solving 
the equation 

^ (a' i+l u k ) 2 
m — cii+i.i+i = 2-> ~ — 

for the i + 1 values of m which satisfy this equation. 

The eigenvector y fc (fc = 1, 2, • • •, i + 1) of Ai +1 corresponding to /x& 
(fc = 1, 2, ■ ■ •, i + 1) may be given by 

v k = (U(n k I - K)- l U'a i+l , 1), fc = 1, 2, • • -, i + 1, 

where U = (u 1} u 2> • • -, U{), 



A = 



Xi 



X 2 



XJ 



and U(nkl — A) -1 C/'ai +1 give the first i components of v k - 

Starting with the matrix (a n ), which has an eigenvalue of an and an 
eigenvector 1, yields the eigenvalues and eigenvectors of the matrix 



a n a i2 

a 21 a 22. 
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and continuing step by step, finally yields the eigenvalues and eigenvectors 
of the matrix A itself. It should be observed that it is necessary to calcu- 
late the eigenvalues and eigenvectors of each of the submatrices A{ (i = 2, 3, 
•••,w— 1) as well as of the matrix A itself. 

Triple Diagonal Method. Let A be a real symmetric matrix. The 
method consists of first reducing the matrix A to a triple diagonal form by 
means of a specially formed orthogonal transformation to be described be- 
low. Then the eigenvalues of the resulting matrix S, which are the same 
as those of A, are obtained with the aid of a Sturm sequence of functions 
consisting of the determinants of the first principal minors or upper left- 
hand corner minors of the matrix X7 — S. Then also the eigenvectors of $ 
associated with an eigenvalue X are obtained directly from the solution of 
the homogeneous equations (X7 — S)x = because of their exceedingly 
simple form. From the eigenvectors of S, one then constructs the eigen- 
vectors of A itself. 

1. In the triple diagonal form of a matrix each element not on the main 
diagonal, the diagonal just above it, or the diagonal just below it is 0. To 
obtain this form one attempts by appropriate orthogonal transformations 
to reduce to all elements of the first row beyond the second column and 
likewise all elements of the first column beyond the second row. If all 
these elements are already 0, no manipulation is required. If not, one next 
looks at the element a\ 2 . If «i2 = and a,\j is the first nonzero element of 
the first row following ai 2 , interchange the second and yth columns and do 
likewise with the second and jth. rows. Thus the new element in the first 
row second column is nonzero. If a 12 t^ to begin with, look at the ele- 
ment ai3. If a 13 = 0, make an exchange similar to the one above so as to 
bring a nonzero element into its position. If a 13 ^ 0, postmultiply A by 
the orthogonal matrix R 23 and premultiply A by R 2 3~ 1 = R'23, where 







(19) E 23 = 1 • • • 





1 











••• 





c 


— s 





••• 





s 


c 





••• 











1 


••• 














1 ••• 














••• 



lj 



u 23 ■ 

L0 7 n _ 3 . 



and 



c = 



1 + 



\a l2 / 



-H 



\a 12 / 



s = \ — ]c. 

\d\ 2 i 



This amounts to a rotation in the x 2 x 3 -Tp\ane, and c, s are the cosine and 
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sine, respectively, of appropriate angles for making the element a\z of the 
matrix R 2S _1 AR 23 = {a' a) equal to and hence also making a' 31 = 0. 
Also a'i2 has larger magnitude than a 12 , and so a' 12 ^ also. Furthermore, 
a\j = aij and a'n = a^ (i, j = 4, • • • , n). If a' 14 ^ 0, one may inter- 
change the third and fourth columns and the third and fourth rows of 
(a\j) and apply the same type of transformation as before, and give rise 
to an additional in the first row and column of the newly formed matrix. 
If a'14 = 0, one looks at a\ 5 , etc. By continuing in this fashion one forms 
a new matrix whose first row and first column, except possibly for the first 
two elements in each case, consist of zeros. 

2. The same scheme can next be applied to the resulting submatrix of 
order n — 1 in the lower right-hand corner. Here instead of R 2 z one uses 
2?34 where 

10 



(20) 



J?34 = 



C/34 







Jn— 4- 



to reduce all elements in first row and column of the (n— l)-st order sub- 
matrix to zero except possibly for the first two elements. No elements 
in the first row or column of the nth order matrix are affected by this. 
Continue in this way and, if necessary, finally use 



R, 



-1.71 — 



.0 Un—\, ri- 

tO effect the final reduction to the following triple diagonal form. 





"ai 


61 








• 









61 


a 2 


b 2 





• 












b 2 


«3 


h 


• 







s = 








bs 


a 4 


&4 • 
























b n —2 


a n— 1 b n _i 




.0 

















b n —l O-n 


It follows that 














(21) 








s 


= T'AT, 





where T consists of a finite product of orthogonal matrices of the type eqs. 
(19), (20), etc., and also of the type obtained from interchanging two col- 
umns of the identity matrix. 
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3. If any b{ = 0, the eigenvalues and eigenvectors of S can be obtained 
by a consideration of the eigenvalues and eigenvectors of each of the two 
individual submatrices thus formed: one above and to the left of the van- 
ishing hi, and the other to the right and below, and both of lower order than 
S. Further, such subdivisions or simplifications are possible as several 
additional b's may vanish. It will suffice then to treat the case b{ 5^ 
(* = 1, 2, .-,n- 1). 

4. Let 

Po(X) = 1 

(22) Pj(X) = X - 01 

Pi(\) = (X - fi^-iPO - & 2 ;_iP;_ 2 (X) (i = 2, 3, • • •, n). 

By expanding the determinant of the first principal minor of X7 — S, whose 
order is i, in terms of the ith. row and zth column, one obtains the last line 
of eq. (22). 

If b{ 9^ (i = 1, 2, • - -,n — 1), the polynomials P n (X) = det \\I — S\, 
P n _i(X), • • •, Pi(X), Po(X) = 1 form a Sturm sequence. This means that 
the eigenvalues of S are distinct and may be isolated. Suppose that c < d 
are two real numbers which are not roots of P n (X). Then the number of 
variations in sign of P n (c), P n _i(c), •• •, Pi(c), 1 minus the number of 
variations in sign of P n (d), P n —\{d), • • •, P\{d), 1 yields the exact number 
of eigenvalues of A between c and d. 

5. Once an eigenvalue X is determined, the homogeneous equations 
(XI — S)x = can be solved to obtain the associated eigenvalue x. The 
equations when written out have the form : 

x 2 = I/61 (X — a^Xi 

X3 = 1/6 2 [(X - a 2 )x 2 - Mi] 



(23) 



l/6,-_i[(X - a;_iK-_i - bi_ 2 Xi_ 2 ] (i = 4, • • •, n) 



= (X — a n )x n — & n _i& M _i. 

It follows from eqs. (23) that if xi is taken as an arbitrary nonzero real 
number that x 2 , x 3 , • • •, x n can be obtained in turn. The last equation of 
(22) may be used as a check. When this has been done for each of the X's 
and one has all the eigenvectors of S, one must turn attention to finding 
the eigenvectors of A. 
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6. Sx = Xx by virtue of eq. (21) implies (T'AT)x = Xx or A(Tx) = 
X(Tx). Hence Tx, where x is an eigenvector of S associated with X, is the 
eigenvector of A associated with the eigenvalue X. 

Adjoint X/ — A and Eigenvectors. Here one assumes that the 
eigenvalues X t - (i = 1, 2, •••, n), not necessarily distinct, have already 
been found. The adj (XI — A), its derivative, or perhaps one of its higher 
derivatives when evaluated at X = \{ present fertile territory for finding 
the eigenvectors associated with X;. 

The adj (hi — A) is a matrix whose elements are polynomials in X but 
may also be viewed as a polynomial in X with matrix coefficients. If one 
writes 

F(X) = adj (X/ - A) = F X n ~ l + F x \ n - 2 + • • •+ F n _ u 

from 

F(X)(XI -A)= f(X)I = lX n + d/X"- 1 +---+c n I 

one may determine the matrix coefficients F 0) F\, • • •, F n _ 1 by expanding 
and comparing coefficients of X. These are 

F = I 

Ft = F A + cj 

F 2 = F t A + c 2 I 



F n _i = F n _ 2 A + c n _J. 



If Xi is a simple root of /(X) = 0, then F(Xi) is of rank 1, and a nonzero 
column of F(Xi) is an eigenvector of A associated with X;. There exists in 
this case only one linearly independent eigenvector of A associated with X;. 
If Xi is a root of /(X) = of multiplicity 2, there can exist two linearly in- 
dependent eigenvectors of A associated with Xi. But this need not be the 
case, as there may exist only one linearly independent eigenvector asso- 
ciated with Xi. In the latter case F(Xi) again is of rank 1 and any nonzero 
column of F(Xi) is an eigenvector associated with X{. On the other hand, if 
there exist two linearly independent eigenvectors associated with Xf, F(X{) 
turns out to be the zero matrix. But F'(Xi), the derivative of F(X) at Xi, is 
of rank 2 and any two linearly independent columns of F'(Xi) are such 
eigenvectors associated with X{. Likewise, if X{ is a triple root of /(X) = 0, 
there can be three linearly independent eigenvectors associated with X», but 
here again this need not be the case. There may be only two linearly inde- 
pendent eigenvectors or even only one. Again if only one linearly inde- 
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pendent eigenvector is associated with \ it any nonzero column of F(X{), 
which has rank 1, is the desired eigenvector associated with A;. If there are 
two linearly independent eigenvectors associated with X;, F(Xi) = and 
F f (\{) is of rank 2, and any two linearly independent columns yield the 
desired eigenvectors. Lastly, if there are three linearly independent eigen- 
vectors, F(\i) = F'(Xi) = 0. But F"(\i) is of rank 3, and any three linearly 
independent columns of F"(Xi) are the desired eigenvectors associated with 
Xi. This procedure can be extended all the way to a root of f(X) = having 
multiplicity n. 

Indirect Methods 

Here the number of arithmetic operations necessary to arrive at exact 
answers is infinite. The procedures are iterative and the eigenvalues and 
eigenvectors of a matrix A are found without explicitly calculating the 
characteristic polynomial of A. 

Iterative Procedures for Hermitian Matrices. It is easier to handle 
the case of a Hermitian matrix since it has real eigenvalues; eigenvectors 
associated with distinct eigenvalues are mutually orthogonal ; and, because 
it can be diagonalized, the multiplicity of each eigenvalue X equals the 
number of linearly independent eigenvectors associated with X. 

Assume, for the time being, that the eigenvalues of a given Hermitian 
matrix A are distinct. Also all the eigenvalues of A + nl, which is also 
Hermitian, can be made positive by picking /x sufficiently large so that there 
is no restriction in assuming that the matrix A has a single dominant eigen- 
value, i.e., an eigenvalue whose absolute value is greater than that of any 
other eigenvalue of A. One first concentrates attention upon a method of 
finding the dominant eigenvalue and its associated eigenvector. Several 
methods are then available for finding the remaining eigenvalues of A. 

The procedure starts with an initial vector x and by repeated premulti- 
plication of A builds up the sequence of vectors 

(24) x p = Ax p _i = A p x (p = 1, 2, • • •)• 

In the nonexceptional case for p sufficiently large, the direction of the 
vector x p will approach the direction of the eigenvector u x associated with 
the dominant eigenvalue Xi. In the exceptional cases, x p will approach 
either some U{ associated with the eigenvalue X{ (i = 2, • • • , n) or else 0. 
The latter rarely happens, but at any rate, an x can be easily picked so 
that the former case will apply. Again if p is sufficiently large, the ratio 
of the ith. component (i = 1, 2, • • •, n) of x p+1 to that of x p can be made 
arbitrarily close to the dominant eigenvalue. The closeness with which 
these ratios agree may be regarded as a measure of the accuracy of the 
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approximation to Xi. An error made during the course of the computation 
of x p will not lead to an erroneous result since subsequent multiplication 
by A will pull the computation back into line. 

One may alternatively calculate Xi by means of a ratio of numbers as 
defined below. Let 

a p = x' Xp (p = 1, 2, •••); 

then 

. ,. a P+i 
Xi = hm 

p— >00 dp 

If next one desires to find the minimum eigenvalue X n of A together with 
its associated eigenvector, one may consider the matrix cl — A where 
c > Xi. The matrix cl — A is Hermitian, and the same techniques may 
be applied to it to find its maximum eigenvalue and its associated eigen- 
vector. To get the minimum eigenvalue X n of A, one simply changes the 
sign of the maximum eigenvalue of cl — A and adds c. The eigenvector 
associated with the maximum eigenvalue of cl — A is also the eigenvector 
associated with the minimum eigenvalue X n of A. 

After Xi and u x have been calculated, the determination of the remain- 
ing n — 1 eigenvalues and their associated eigenvectors of the nth order 
matrix A may be done in terms of a matrix whose order is n — 1 instead 
of n. If the normalized form of Ux is denoted by u{*, i.e., U\* = U\/\ui\, 
from the vector U\ * a unitary matrix U may be constructed so that 



U'AU = 



Xi 
LO Al 



where A x is a Hermitian matrix of order n — 1 whose eigenvalues X 2 , X 3 , 
• • •, X n are the n — 1 remaining unknown eigenvalues of A. The dominant 
eigenvalue X 2 of A i and its associated eigenvector v 2 can be found as pre- 
viously. The eigenvector u 2 * associated with the eigenvalue X 2 of A is the 
vector U („°). The following construction of the unitary matrix U is due 
to Feller and Forsythe. One writes U\* as follows 

t*i* = C), 

where a is a complex number and z is an n — 1 dimensional vector with 
complex components. Then 



U = 
where k = (1 — a)/{\ — ad). 



a —z 

.z I n —\ — kzz'_ 
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Next one replaces Ai by the matrix 

"X 2 o ■ 
.0 A 2 _ 

where A 2 is a Hermitian matrix of order n — 2 having eigenvalues X 3 , • • • , 
X n , and then one repeats the previous step. This is continued until all 
eigenvalues and eigenvectors of A are obtained. 

Another way to find the eigenvalues X 2 , • • • , X n and their associated 
eigenvectors, once Xi and u x * are known, is to form the new Hermitian 
matrix 



(25) . A x = A - XiW!*u*' 



of order n also. The eigenvalues of A x are 0, X 2 , • • • , X n . The known eigen- 
vector Ui* is associated with the eigenvalue of A x ; while the unknown 
eigenvector U{* is associated with the eigenvalue X; (i = 2, • • •, n) of A x as 
well as of A. Thus the dominant eigenvalue X 2 oi A± and its associated 
eigenvector u 2 * can be found as before by forming powers of A x instead of 
powers of A as in eq. (24). 

Next one forms the Hermitian matrix 

A 2 = Ai — \ 2 u 2 *u*'2 

of order n which has eigenvalues 0, 0, X3, • • •, X n , and the unknown u { * is 
associated with the X; (i = 3, • • •, n) of A 2 as well as of A. Thus one ob- 
tains X 3 and its associated eigenvector u 3 *. Again one continues in this 
fashion until all eigenvalues of A and their associated eigenvectors are 
found. 

Multiple Roots. So far the possibility of multiple roots has not been 
considered. Suppose, as before, one starts with x and builds up sequence 
(24), one obtains as before an eigenvector associated with Xi. A distinct 
starting vector y may be selected to build up a new sequence which will 
again lead to the eigenvalue Xi. But it may happen that y leads to an 
eigenvector which is linearly independent of the one to which xq leads. In 
this case Xi is a multiple eigenvalue. If y , as in the case of distinct eigen- 
values, leads only to an eigenvector which is linearly dependent or simply 
a multiple of the eigenvector to which x leads, then X x is a simple eigen- 
value. If xq and y lead to linearly independent eigenvectors, and a third 
arbitrary vector z leads to an eigenvector which is linearly dependent upon 
the first two eigenvectors, Xi is an eigenvalue of multiplicity 2; whereas, 
if z leads to an eigenvector linearly independent of the first two calcu- 
lated eigenvectors, Xi is at least of multiplicity 3. One can continue this 
process for eigenvalues of higher multiplicity also. 
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Let Xi be a root of multiplicity 2. Then in the two-dimensional vector 
space generated by the two linearly independent eigenvectors obtained one 
may select Ui*, u 2 * t which are orthogonal and of unit length, and which 
are eigenvectors associated with Xi. By starting with Wi* and u 2 * one may 
similarly, as before, build up a unitary matrix U such that 



U~ l AU = 



Xi 
Xi 
.0 A 2 . 



where A 2 is a Hermitian matrix of order n — 2 containing the remaining 
eigenvalues X 3 , • • •, X n of A. Similarly, one proceeds in the case of multi- 
ple eigenvalues, as outlined before. 

A number of additional variations for obtaining the eigenvalues and ei- 
genvectors of A is possible. 

Iterative Process for General Type Matrices. If the matrix A can 
be diagonalized, the method of successively premultiplying by A applies, 
with some small appropriate modifications, to this case as well as to the 
case of the Hermitian matrix. No longer are the eigenvalues of A neces- 
sarily real. There may be several distinct dominant eigenvalues. The 
eigenvectors of A can no longer be assumed mutually orthogonal. In order 
to get around this situation, one introduces the concept of row eigenvectors 
as well as column eigenvectors. Associated with each eigenvalue X t - of A 
is the row eigenvector w (t) , where u^A = X 1 m (i) , and the column eigenvec- 
tor u^ where Au{ = X;w t - (i = 1, 2, • • •, n). In this case 



where 



6v Uij *~ ®%ij 
8{j = 0, 15* j, 

8ij =1, i = j. 

Here w (l) and U{ {i = 1, 2, • • •, n) need not be unit vectors but only 
u w u { = 1 (i = 1, 2, •••, n). 

One again starts with an arbitrary initial vector x and forms the sequence 
x p of eq. 24. A unique dominant eigenvalue and its associated eigenvector 
are found exactly as before. Whereas, in the case of a Hermitian matrix 
one forms the matrix A i as in eq. (25) in order to study the remaining eigen- 
values and their associated eigenvectors, one now forms the matrix 

Ai = A — Xiw (1) wi. 

Finding the eigenvalues in the case where several eigenvalues are dom- 
inant is more complicated, as these are not computed as a simple ratio but 
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rather as described below. Suppose that x ip , x 2p , • • • , xu P are k linearly 
independent vectors obtained from the sequence (24). For p sufficiently 
large, these are arbitrarily close to the actual eigenvectors. It is desired 
to find the eigenvalues X x , X 2 , ■ • •, X& associated with these. Take z as an 
arbitrary vector and form 



Z Xi 



(i= 1,2, ..-,/c), 



then 



det 



1 


dip 


a>2p 


"kp 


X 


a l,p+l 


a 2,p+l 


a k,p+l 


X 2 


a l,p+2 


a 2,p +2 


dk,p+2 


x fc 


Ul,p+k 


a 2,p+k 


O'k.p+k 



= 



has k roots which are close approximations to the eigenvalues X 1} X 2 , • • • , X&. 

The great majority of matrices appearing in applications have distinct 
eigenvalues and so can bediagonalized. Therefore, the method of iterat- 
ing by premultiplication of a given matrix is applicable. In the rare case 
in which A has a root of multiplicity r and whose associated eigenvectors 
number less than the full complement of r, it is not possible to diagonalize 
A. Nevertheless, even in this case, it is still possible to use this iteration 
scheme to find the dominant eigenvalue and the associated eigenvector of 
matrix A having but a single dominant eigenvalue. One must, however, 
consider the linear dependence of a finite number of successive x p s in 
sequence (24) for p sufficiently large to obtain the dominant eigenvalue 
Xi. The associated eigenvector may be obtained as a linear combination 
of a finite number of the x p 's whose components contain powers of Xi. 

Jacobi Method. The technique applies to Hermitian and so to real 

symmetric matrices too. The method hinges on the fact that a 2 X 2 

Hermitian matrix 

ran "^ 
H =\ 

lae~ l 



ae" 



-iip 



a 2 2 



a > 



can be reduced to diagonal form D by a unitary transformation U 1 HU 
= D, where 

re^ /2 cos0 -e^ /2 sin0 " 
- { + 12 sin 6 e-^ 12 cos 6. 



(26) 



U = 



and 6 is an angle in the first quadrant which satisfies tan 29 = 2a/ (a n —a 2 2)- 
If \p = 0, i.e., // is real symmetric, the matrix (26) reduces to the familiar 
form 
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U = 



cos 8 — sin 6 
sin 8 cos 0_ 



which corresponds to a rotation in the plane. 
If the nth order Hermitian matrix A = (a t -y) is written in the form 

H A x ' 

_ > 



A = 



then the unitary matrix 

(27) 



tfi = 



'U 
.0 I. 



where U is the matrix (26) and transforms A into a matrix B = (&#), where 
&12 = &21 = 0; furthermore, the sum of the squares of the diagonal ele- 
ments of B exceeds the corresponding sum of A by the positive quantity 
2a 2 . If it is desired to transform A into a matrix B such that & t -y = &y t - 
= 0, the elements of the matrix U in eq. (26) must be positioned in the 
ith and jth rows as well as in the ith and jth columns of U\ in eq. (27). 

One might hope that after applying the product of a finite number of 
the above unitary transformations one might reduce the matrix A to di- 
agonal form, in which case the sum of the squares of the diagonal elements 
will have the maximum possible value. Unfortunately, this is not true, as 
some of the elements, which have previously been reduced to zero, will not 
remain so while some additional elements are likewise being reduced to 
zero. 

The procedure is to reduce to zero a pair of off-diagonal elements of 
greatest modulus. It is the infinite product of all these transformations 
which will reduce A to diagonal form A and whose diagonal contains the 
eigenvalues of A. The infinite product of unitary matrices of the type 
(27) converges to a matrix whose columns are the eigenvectors of the matrix 
A. 

Eigenvalues of Special Matrices 

The types of eigenvalues to which certain important classes of matrices 
give rise are worth noting. 



Matrix A 

(a) Real and symmetric 

(b) Real, symmetric, and 
positive definite 

(c) Real, symmetric and 
positive semidefinite 

(d) Orthogonal 



Every Eigenvalue A; of A 

(a) Real 

(b) Real and positive 

(c) Real and non-negative 

(d) |Xj| =1 for every i 
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In (a), (b), and (c), if a real symmetric matrix is replaced by a Hermitian 
matrix, the conclusions still remain valid. In (d) if A is unitary, the con- 
clusion drawn there still holds. 

Some additional properties concerning dominant roots of important 
classes are listed below: 

(i) If A is real and symmetric, the maximum eigenvalue X max is given by 

x'Ax 
Xmax = max — — • 

x^O XX 

and the minimum eigenvalue X m i n is given by 

x'Ax 
Xmin = min — - — 



x^o xx 

(ii) If A is a real positive matrix (i.e., A has positive elements), X ma x is a 
real number. 

Bounds on Eigenvalues 

It is often a helpful guide to establish bounds for the eigenvalues of a 
matrix at the outset, as this may influence the procedure. It is extremely 
advantageous when this leads to the isolation of some of the eigenvalues 
of a given matrix. Some of the criteria for determining bounds are easily 
applied. A number of such results will be stated and in some cases addi- 
tional information will be given concerning the associated eigenvectors. 
First, the case of matrices with complex elements will be treated, and sub- 
sequently this will be specialized to matrices with positive and also mat- 
rices with non-negative elements. However, when results on bounds apply 
to a large class of matrices, the bounds cannot be expected to be as sharp 
as those applying to a smaller more specialized class of matrices. The fol- 
lowing cases are of interest. 

1. Let A be an arbitrary matrix of order n with complex elements. Then 

|X| ^n M , 

where X is any eigenvalue of A, and M is the maximum of the moduli of the 
elements a# (i,j = 1, • • •, n) of A. This result is due to Hirsch in 1902. 

2. Let 

n n 

Ri = 2 I a H I an< i jT,- = 23 I a a I • 

Also let R = max R{ (i = 1, • • •, n) and T = max Tj (j = 1, • • •, n). 
Then 

|X| ^ min (R, T). 
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A number of variations on these two bounds (1) and (2) exists. Some 
of these variations are a bit sharper, but these are simple to apply and will 
be sufficient for the purposes at hand. 

3. Let Pi denote the sum of the moduli of the off-diagonal elements of 
the ith. row of the matrix A and Qj the sum of the moduli of the off-diagonal 
elements of the jth. column of A. That is, 

n n 

Pi - 2 \aij\ and Qj = 2 \ a n\- 

Then a result due to Levy and Hadamard states that each eigenvalue of A 
lies in at least one of the circles 

(28) \z - a {i \ ^ Pi (i = 1, -..,n) 

and in at least one of the circles 

\z - ajjl ^ Qj (j = 1, •■•,«). 

In other words, if one takes the diagonal element an and draws' a circle 
with an as the center and Pi (i = 1, • • •, n) as radius, all the eigenvalues 
of A will be trapped in these n circles. A similar remark applies to the n 
circles with the Qj (j = 1, • • • , n) as radii. It is to be noted that an eigen- 
value of A may be in several of the n circles. 

4. An interesting offshoot of this result is the following: If one of the n 
circles is isolated from the remaining n — 1 circles, that is, has no point in 
common with the remaining n — 1 circles, exactly one eigenvalue of A 
will be found in the isolated circle. More generally Gersgorin showed that 
when m circles intersect in a connected region isolated from the remaining 
n — m circles, the connected region thus formed contains exactly m eigen- 
values of A. 

5. The following results concerning the number of associated eigenvectors 
is noteworthy. If an eigenvalue X of the matrix A lies in only one of n 
circles (28), X has only one linearly independent eigenvector associated 
with it. This result is due to Taussky (Ref. 15). Stein has shown that if 
an eigenvalue X has associated with it m 5j n linearly independent eigen- 
vectors, X lies in at least m of the circles (28). 

6. Before passing to the case of positive and non-negative matrices, it is 
worth noting a result of Frobenius which gives a connection between the 
eigenvalues of a matrix with complex elements and a dominating matrix 
with non-negative elements. Let B = (b{j) be a matrix with complex ele- 
ments and A = (a^) be a matrix with non-negative elements such that 
\bij\ ^ a { j (i, j = 1, • - -, n). Then the characteristic circle of A contains 
the characteristic circle of B. (The characteristic circle of a matrix is the 
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smallest circle about the origin containing the eigenvalues of the given 
matrix.) 

7. Turning attention next to real matrices with non-negative elements, one 
can draw some additional and sharper conclusions. If A is a matrix whose 
elements a# ^ (i, j = 1, • • •, n) then (a) A has a real eigenvalue Xd j^ 
which is dominant (there may be other dominant eigenvalues), (6) Xd has 
an associated eigenvector x ^ 0, i.e., all components of x are non-negative, 
and (c) Xd does not decrease when an element of A increases. 

The above results are due to Herstein and Debreu, who paralleled for the 
case of non-negative matrices the results of Frobenius given below. 

8. These results grow sharper when one further restricts the matrix A 
to be indecomposable. A non-negative matrix A is called indecomposable 
if A cannot be transformed to a matrix of the form 

A n A 12 
A 22 . 

by the s*ame permutations of rows and columns where A n and A 22 are 
square submatrices of A. 

If A is a non-negative indecomposable matrix, then (a) A has a real 
simple eigenvalue Xd > which is dominant; (6) Xd has an associated eigen- 
vector x > 0, i.e., all the components of x are strictly positive; and (c) Xd 
increases when an element of A increases. These important results were 
first demonstrated by Frobenius nearly a half century ago. 

9. If the matrix A is still further restricted so that all its elements are 
positive, that is, a^- > (i, j = 1, • ■ • , n), then the statement (a) above 
can be strengthened to include the fact that Xj is the only dominant eigen- 
value of A. 

10. Again, suppose A is a positive matrix and let 

n 

Ri = ]C a u (i = 1, • • •, n), .R = max {Ri, R 2 , ••-, R n }, 

and r = min {i^i, R 2 , • • •, R n }. 
Frobenius first noted that 

r S X d ^ R. 

Also Xd = r = R if and only if all R { are equal; otherwise, the inequality 

r < Xd <R 

holds. Suppose that not all the R { (i = 1, • • -, n) are equal and let 

k = min a,ij, 

5 = max {Ri/Rj}, 
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and 

a = V(r - k)/(R - k) 

so that k > 0, 5 < 1, and a < 1. Ledermann improved the bounds on 
Frobenius' result as follows 

r + K (v F ~ / <Xd <E-K '( 1 ~ v ^)> 

and Ostrowski further sharpened the bounds with the inequalities 

r + k(- - l) £\ d £ R - k(1 - a). 

In fact, the right-hand side of Ostrowski's inequalities applies to matrices 
with complex elements when in the definitions of R and k one uses the 
modulus of the elements. More recently, Brauer announced further im- 
provement of the above bounds, stating that the best possible bounds have 
been attained. That is, in order to get sharper bounds one would have to 
restrict further the class of positive matrices. 

11. Some specialized examples of non-negative matrices are the stochastic 
matrices and the oscillation matrices. The eigenvalues of the former play 
an important role in the theory of stochastic processes while the latter type 
matrices are applicable in the theory of small oscillations of mechanical 
systems. 

The matrix A = (an) is called stochastic if a# ^ (i, j = 1, • • •, n) and if 

n 

X) an =1 0" = 1, •••, n). 

y=i 

If aij > (i, j = 1, • • •, n), the matrix A is called a positive stochastic . 
matrix. All the eigenvalues of a stochastic matrix lie within or on the 
boundary of the unit circle. Also X = 1 is a dominant eigenvalue of any 
stochastic matrix. Previous results on non-negative and positive matrices 
may be directly applied to stochastic matrices. 

The matrix A of order n is said to be completely non-negative (completely 
positive) if all minors of all orders from 1 to n of A are non-negative (positive). 

If A is completely non-negative and there exists a positive integer k such 
that A k is completely positive, then A is said to be an oscillation matrix. A 
non-negative matrix A will specialize to an oscillation matrix if and only if 
det A 9± 0, a;,i + i > and a t - + i,i > (i = 1, • • •, n — 1). The eigenvalues 
of an oscillation matrix have the interesting property that they are all 
strictly positive and simple. 

For an extensive bibliography on the bounds of eigenvalues, see Ref. 15. 
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Table 6. Computer Storage Requirements and Number of Operations for 
Finding Eigenvalues and Eigenvectors 

n = order of matrix involved 
w, w', w" = program storage requirements 



Method 


Storage 
Requirements 

Triple Diagonal 


- Multipli- 
cations 


Addi- 
tions 


Triple diagonal form S 
Eigenvalues of A a 
Eigenvectors of S 
Eigenvectors of A 
Total for eigenvalues 
and eigenvectors 6 


n 2 /2 + n/2 + w 

2n-l + w' 
2n 2 + w" 

f n 2 + -fn — 1 -f- w'" 


3n 2 
2n 3 


%n z 

2n 2 
n 3 

fn 3 


Jacobi (Symmetric matrix) 






Eigenvalues 

(one step of reduc- 
tion) 


n 2 + 4 + w 


4n 


2n 



° In finding the eigenvalues of A, the number of operations depends on the stipulated 
requirements for accuracy. If n is sufficiently large, the number of operations required 
to find the eigenvalues, once the matrix is in triple diagonal form, is negligible compared 
with the number of operations required to reduce the original matrix to triple diagonal 
form. 

b id'" is the sum of w, w', w", and the number of cell locations used in finding the eigen- 
vectors of A. 



4. DIGITAL TECHNIQUES IN STATISTICAL ANALYSIS 

OF EXPERIMENTS Joseph M. Cameron 

Introduction. In scientific experiments a variable is measured under 
several different conditions with a view to assessing the effect of these con- 
ditions on the variable under study. There may be factors present in the 
measurement process which, if not balanced out or their effect reduced by 
randomization or replication, may invalidate the estimates of the effects 
the experiment seeks to measure. The branch of statistics called the design 
of experiments is concerned with the construction of experimental arrange- 
ments that permit the balancing out of such extraneous factors and at the 
same time minimizing (for a given number of observations) the uncertain- 
ties in the estimates of the effects under study. 
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In most applications the analysis required is the usual least squares anal- 
ysis for estimating the parameters postulated to represent the data. In a 
designed experiment the normal equations that arise in the estimation of 
the parameters take on a particularly simple form and the calculations 
have been systematized and given the name analysis of variance. Example. 
Consider a set of measurements x\, x 2 , • • • , x n all postulated to be estimates 
of a single quantity. The least squares estimate for that quantity is, of 
course, the average x = Hxj/n. One can also compute from the data a 
measure of the dispersion of the results about this average. Perhaps the 
most common such measure is the standard deviation, v 2(x; — x) 2 /(n — 1). 
In the analysis of variance one deals not with the standard deviation but 
rather with its square, which is a quadratic form in the deviations divided 
by the number of independent deviations, called the number of degrees of 
freedom. 

The analysis of variance in its general form is a technique for (a) com- 
puting estimates of the parameters involved in the problem and (6) com- 
puting the value of quadratic forms, called sums of squares, assignable to 
certain groupings of the parameters, each sum of squares carrying with it a 
certain number of degrees of freedom (the rank of the quadratic form). 
Thus in the case of k averages each based on n measurements, the param- 
eters to be estimated are the grand average and the (k — 1) independent 
deviations of the individual averages about this grand average. Three 
sums of squares are to be calculated : one for the grand average (with one 
degree of freedom), one for the deviation of the individual averages about 
the grand average [with (k — 1) degrees of freedom], and one for the devia- 
tions of the observations about their own group averages [with k(n — 1) 
degrees of freedom]. 

Several examples of the analysis of variance are presented to illustrate 
the different techniques of computation that are available. The advantage 
of one over another probably depends on the nature of the computing de- 
vice used. 

The availability of modern high-speed digital computers makes it feas- 
ible to analyze experimental data involving a much greater number of 
factors, each factor occurring at more levels than would otherwise be the 
case. The types of calculations described above and in the succeeding 
pages, because of their systematic nature, lend themselves particularly well 
to treatment on automatic digital computers. 

Analysis of Factorial Designs Using Hartley Method. An experi- 
ment in which the effects of several factors on a variable are studied by 
making measurements at all possible combinations of the several states or 
levels for each of the factors is called a factorial experiment. Example. 
Four temperatures of heat treating can be combined with three time periods 
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to give rise to twelve conditioning treatments for some alloy. This would 
be a factorial design with two factors, one at four levels and the other at 
three levels. 

The most general method for the analysis of factorials was developed by 
Hartley (Ref. 22). His method depends on three operators which he has 
labeled 2, D, and ( ) 2 , defined as follows: 

S< Sum over all levels t = 1, 2, • • •, T for each combination of the 

other subscripts. 
D t Difference between T times the original values and the total in 
the set 2* to which the original value contributed. 
( ) 2 Sum of squares of items indicated in the parentheses. 

Procedure. The use of this technique will be illustrated for a two-factor 
factorial having one factor at k levels and the other at n levels. Denote 
by Xij the observation at the tth level of the first factor and jth. level of the 
second factor. Let xj denote the set of sums ^ x a = x .ji there being n 

i 

such sums. In Table 7 the plan of the calculations is shown. Table 8 
shows the analysis of variance table derived from the results of Table 7. 

Table 7. Plan op Calculations Using Hartley Technique, Two-Factor 

Factorial Experiment 







A i 




A 2 




Bi 


x n 




Xn 


Level of fac- 


B 2 


Xu 




Z22 


tor B 




... 








B n 


X\n 




Xln 




Sy 


Xl 




Xi. 






nxn — 


Xl. 


nx 2 i — x 2 




Dj 


nxi2 — 


xi. 


nx 22 — x 2 



DiDj 



nxi n — x\. nx 2n — x 2 . 

kxi. — x.. 
knxn — kxi. — nx.i + x. 

KtlXin ' ftXi, TlX.n ~\~ X.. 



Xkl 




Xk 2 




Xkn 




Xk. 




nxki — 


Xk. 


nxki — 


Xk. 


ilXkn 


Xk. 



2,-2y 
S*Z)y 



nx.i — x. 
nx. 2 — x. 



I IX .71 X* . 



l£Xk. X.. 

knxn — kxk. — ■ nx.i + x.. 
knxkn — kxk. — nx.n + x.. 



The estimates of the parameters are obtained by dividing the entries in 
the sets 2j2y, 2;Z)y, D;2y, and DiDj by nk giving in that order the grand 
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Table 8. 


Anam 


sis of Varianci 


3 for Two-Factor 


Factorial Experiment 




No. of 


Sum of 


Degrees of 


Sum of Squares Is 




Items 


Squares 


Freedom 


Associated with: 


(2<Sy) 2 


1 


&&)*/nk 


1 


Grand average 


ViDj) 2 


n 


(PiDjyMnk) 


n — 1 


Effect of different levels 
of factor B 


(DZj) 2 


k 


(D&y/Knk) 


ft- 1 


Effect of different levels 
of factor A 


(Dpy 


nk 


(DiD^/ink)* 


(ft - 1)(* - 1) 


Interaction: lack of con- 
stancy between levels 
of A as level of B is 
varied 






2j2jXij 


nk 


Total (for check) 



average, differences among levels of factor B, differences among levels of 
factor A, and the differences due to lack of constancy of the different 
levels of factor A as the level of factor B is changed. This technique can 
be extended to cover the case of three or more factors by using the basic 
operations of 2, D, or ( ) 2 and is adaptable to other designs as well (see 
Ref. 22). 

An alternate procedure necessary when the experiment is run in blocks 
containing only a fraction of the total number of observations or when a 
fractional replication design is used is based on the technique described in 
Ref. 23, and is discussed below. Still another procedure is given in Ref. 16 
based on the computation of individual degrees of freedom with orthogonal 
polynomials tabled in Refs. 20 and 21. 

Balanced Incomplete Blocks. When there are more objects or treat- 
ments than can be compared under the same conditions, i.e., on a given 
batch of material, in a given time period, or other factor which limits the 
uniformity of conditions to a few tests, it is necessary to schedule the 
measurements so that all comparisons of interest may be estimated from 
the data. The class of designs constructed for such a case is called incom- 
plete block designs, the block being the group of tests within which the 
environmental or other factor is assumed not to change. The analysis of 
these block designs will be illustrated for the case of the balanced incom- 
plete block design (see Refs. 16-23). 

Observations have index Xbktr referring to B blocks with K units per block 
and T treatments with R repetitions of each. The data are entered so 
that the observations from the first block come first, followed by those 
from the second block, etc. 

Step I. Compute total sum of squares of original values, 2z&fc 2 . 
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Step II. Consider only indices b and k. 





Number 




( ) 2 Applied to 


Operation 


of Items 


Result 


Result Gives BK Times 


2* 


B 


Xb. 




D k 


BK 


Kxbk — Xb. 




2&2fc 


1 


x.. 


Correction factor 


Db2 k 


B 


Bxb. — x„ 


Unadjusted blocks sums 
of squares 



Step III. Now consider only indices t and r. The values of Dk are now 
rearranged into T groups with R values each so that the R values corre- 
sponding to the first treatment come in a group followed by a similar 
grouping for each of the remaining treatments. Call these values d tr and 
denote operations after rearrangement with asterisk. ~2 r *Dk results in 

d t . = Kx t . — B t 

. K(K - 1)TR 
(Z r *Dk) = X sum of squares for treatments (adjusted), 

where B t = sum of block totals for blocks containing treatment t. 



Analysis of Variance 

Total 

Blocks (unadjusted) 

Treatments (adjusted) 

Error 



Sum of Squares 

Zxbk 2 - x\./BK 

(DbZtf/BK 

(T - l)(Z r *Dk) 2 

K(K - \)TR 
By subtraction 



Degrees of Freedom 

BK-1 

B - 1 

T - 1 
BK - B - T + 1 



Analysis of Factorials by Using Relations among the Indices 
Associated with the Treatments. To illustrate the method assume 
there are three factors A, B, and C having levels n + 1, n + 1, and n + 1 
respectively. Each observation is tagged with an index X\X 2 Xz, where X\ = 
0,1, ■ • •, n, x 2 = 0, 1, ■ • •, n, and x 3 = 0, 1, • • •, n, where n is a prime. 

For the main effect of A form the (n + 1) sums of values whose indices 
satisfy 

xi = mod (n + 1) 

Xi = 1 mod {n -f- 1) 



X\ = n mod (n + 1) 
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Denote these sums by A x , A 2 , • • • , A n+1 . The sum of squares for the main 
effect of A is given by 

SA 2 (SA) 2 



(n + l) 2 in + 1) J 



(degrees of freedom = n). 



Similar computations give the sum of squares for the main effects of B 
and C. 

For the two-factor interactions the sums of values whose indices satisfy 
the equations below are computed. 

Xi + x 2 = mod (n + 1) 



_X\ -f- x 2 = n mod (n + 1) 



x\ + nx 2 = mod (n + 1) 



_Xi + nx 2 = n mod (n + 1) 

From the (n + 1) sums corresponding to X\ + ax 2 — 0, 1, • • •, n mod 
(n + 1) are computed the sum of squares associated with the n degrees 
of freedom for AB a and the AB interaction is given by the total of such 
sums over all values of a. For the three-factor interaction one computes 
the (n + l)n 2 sums of values for which the indices satisfy 

Xi + ax 2 -f j&c 3 = 0, 1, - • -, n mod (n + 1), 

where a = 1, 2, • • •, n, and /S = 1, 2, • • •, n. Each group of (n -j- 1) 
sums give the sum of squares associated with the n degrees of freedom for 
the effect AB a C^. For each group one computes: 



S (Sums)' 



(Grand total) s 



Number of items in each sum Total number of items 

The extension to higher order interactions is straightforward. 

This technique is ideally adapted to analysis of variance of factorials 
where block confounding occurs or to the analysis of fractional replication 
of factorials. Example. A 3 4 design in blocks of 9 with ABD, ACD 2 , 
AB 2 C 2 , and BC 2 D 2 confounded with blocks is computed in the manner 
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described to get the usual analysis except for the combination of the sums 
of squares for the three-factor interactions which involve confounding with 
blocks. For example the ABD interaction is given by the sum of squares 
associated with AB 2 D, ABD 2 , and AB 2 D 2 each of which has two degrees 
of freedom. The sum of squares associated with ABD is assigned to 
blocks. 

For fractional factorials (with or without block confounding) the analysis 
is carried out as if it were a complete design with fewer factors by sup- 
pressing one or more of the indices. The individual components, A, • • •, 
B, • • • , AB, AB 2 , • • • , ABC, ABC 2 , • • • , are computed, and an identifica- 
tion is made according to the identity relationships (and block confounding, 
if any). (For further details see Ref. 23.) 

Analysis of Variance for 2 n Factorials. An example for a 2 2 experi- 
ment will illustrate this procedure. Enter observations in the order desig- 
nated. 



Observed First Sums and 

Values Differences, Di 

(1) = zoo (1) + a 

a = xoi b -\- ab 



Second Sums and 
Differences, D 2 

(1) + a + b + ab 
a - (1) + ab - b 



D 2 2 /2 n Will Give 

Correc. for mean 
A 



b = Xio a — (1) b + ab — (1) — a B 

ab = xu ab — b ab — b — a + (1) AB 

In general: 

(a) Form a column of sums of the 2 n ~ 1 pairs followed by 2 W—1 differences 
between the first and second element of a pair. 

(6) Repeat this operation on the column so formed until the nth such 
column is formed. 

(c) Then square the entries in the nth column and divide by 2 n to get 
analysis of variance table in the order A, B, AB, C, AC, BC, ABC, 
The observations are entered so that their subscripts form an increasing 
sequence when regarded as binary numbers; e.g., for n = 3 the observations 
are in the order z oo ^ooi ^oio ^on ^ioo ^101 #no ^ni- 

Analysis of Fractional Replication of 2 n Factorials. Arrange the 
(l/2* : )2 n = 2 s observations in ihe proper order for a 2 s factorial (suppress- 
ing the other indices) and carry out the analysis as above. Identify the 
results of the analysis by using the identity relationships and the block 
confounding in the manner shown in the following example. 



Example. }/i replication of 2 6 in blocks of 8. 

Fundamental identity: / = ABEF = ACDF = BCDE. 
Block confounding: CD. 
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Block 


Treatment 


Index ° 


Identification 


1 


(1) 


0000 00 


Mean 


1 


of 


0001 01 


A = A 


1 


6e 


0010 10 


B = B 


1 


abef 


0011 11 


AB = AB + EF 


2 


cef 


0100 11 


C = C 


2 


ace 


0101 10 


AC = AC + DF 


2 


bcf 


0110 01 


BC = BC + DE 


2 


abc 


0111 00 


ABC = error 


2 


def 


1000 11 


D = D 


2 


ade 


1001 10 


AD = AD + CF 


2 


bdf 


1010 01 


BD = BD + CE 


2 - 


abd 


1011 00 


ABD = error 


1 


cd 


1100 00 


CD = CD + AF + BE + blocks 


1 


acdf 


1101 01 


ACD = F 


1 


bcde 


1110 10 


BCD = E 



1 abcdef 1111 11 ABCD = AE + BF 

a Only the first four indices are used. 



5. ORDINARY DIFFERENTIAL EQUATIONS Richard F. Clippinger 

Definitions and Introduction. An ordinary differential equation of 
nth order is a relation between an independent variable x, a dependent 
variable y\, and derivatives of y\ up to order n, (d n yi/dx n = ?/i (n) ): 

F(x, yi (x),y\(x), •■• J y 1 (B) (aO) =0. 

By the introduction of new variables, it is possible to obtain a system of n 
equations of first order: 

Gi(x, 7/i (x), y 2 (x), • • -, y n (x), y'i(x), • ■ •, y' n (x)) = 

G 2 (x, y x (x), y 2 {x), •■-, y n (x), y\(x), ■•-, y' n (x)) = 



G n (x, yi(x), y 2 (x), • • •, y n (x), y\(x), • • •, y' n {x)) = 
which theoretically can usually be solved in the form : 
y'i = fi(x, yi(x), ••-, y n {x)). 



y'n = fn(x, yi(x), ■••, y n (x)). 
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With vector notation, this system takes the form: 

(29) y'{x) = f(x,y(x)), 

where y is a vector whose components are yi(x), i = 1, 2, • • • , n, and / is 
a vector whose components are fj(x, y\{x), • • •, y n (%)), j = 1, 2, • • •, n. 

Vector notation will be used throughout this section covering systems of 
equations which can be put in this form. The reader who is not familiar 
with vectors can take the case where y(x) is a single function of x and use 
this section as a guide to the solution of one first order equation. 

At the end of this section is a summary table of some useful numerical 
methods for solving differential equations on a digital computer (see Table 
11). Some important characteristics of each of these methods are listed. 
The prospective user may employ this table as a quick guide in selecting 
the most suitable method for the problem at hand. 

Requirements for Solution. A solution of eq. (29) is a vector y(x) 
which satisfies eq. (29). It necessarily possesses a first derivative. 

The differential equations used by engineers nearly always possess solu- 
tions which have continuous derivatives of many or all orders or indeed 
are analytic (i.e., the Taylor series converges) except at isolated points. 
They are said to be piecewise continuous and have piecewise continuous 
derivatives. The isolated discontinuities are of practical importance since 
engineer's derivatives are such quantities as current, voltage, velocity, and 
acceleration which he must limit to avoid damage to his equipment. Meth- 
ods of solving differential equations that are awkward at discontinuities 
are of restricted value to him. 

Numerical Solution. The Taylor series for y{x) in the neighborhood 
of some point x : 

y(x) = y(x ) + y'(x )(x - x ) -\ h y (m) (x )(x - x ) m /m\ ^ , 

enables one to approximate y by an rath degree polynomial in x — x . 
Most numerical methods of solving differential equations depend directly or 
indirectly on this fact. 
Consider a set of points 

Xi + j = Xj -+- ih, i = 0, ±1, ±2, 

These points are equally spaced along the rc-axis and the distance between 
neighboring points is h, called the grid size. 

Write the Taylor series of y, hy', h 2 y" , etc., at each of these points: 

(30a) y i+j = y(x i+j ) = Vj + ihy'j + • • • + i m h m y/ m) /ml + R m+1 , 

(306) hy'i+i = hy'j + ih 2 y/' +■■■+ i m - l h m y^/(m - 1) ! + R m+h 

(30c) h 2 y" i+j = h 2 y"j + • • • + i m - 2 h m yj M /(m - 2) ! + R m+1 , 
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where R m+ i is a generic notation for a remainder which contains h m+1 as a 
factor. Equations (30) can be used in an endless variety of ways to obtain 
procedures for the numerical solutions of eq. (29). 

Solutions for Known yi andy';. ij{ and y'i are known at several past 
points (i.e., i = 0, — 1, — 2, • • •, — I) and yj +1 is desired. Solve eqs. (30a) 
and (306) at i = — 1, • • •, — I for 21 of the quantities: 

h\-/2\, *V 3) /3!, • ■ ■, /> 27+ V 27+1 )/(27 + 1)!, 

and substitute into eq. (30a) for ?/y +1 and obtain a formula accurate to 
terms of degree 27 + 2 in h. Thus we have Table 9. 









Table 9. 


Extrapolation Formulas 






For- 
mula 


I 


2/J+1 


Vi 


hy'j 


j/y-i 


hy'j-i yj-2 hy'j-2 yj-3 


hy'j-3 


Error 


1 
2 
3 

4 



1 
2 
3 


1 
1 
1 

1 


l 

-4 
-18 

128 
— 3 


1 

4 

9 

16 


5 

9 

-36 


(Euler method) 
2 
18 10 3 
72 64 48 ^f 


4 


!/ (2 >A a /2 

2/ (4) fc 4 /8 

/>V 6) /20 

/>y 8 >/70 



First Order Method. Formula 1 of Table 9 is the simplest and best known 
of all solution methods and is due to Euler. The value of yj + i is 

(31) 2/y+i = Vi + hy' jf 

then the value of y'j+i is obtained from eq. (29). It can be shown that the 
approximate solution obtained in this fashion converges to the exact solu- 
tion as the grid size approaches zero, the error at a given point being propor- 
tional to h. This is called a first order method. The principal attraction of 
this method is its simplicity. Its principal disadvantage whether for hand 
or electronic digital computation is that it requires a small grid size to ob- 
tain a given accuracy. 

Studying the Stability of the Method. The most illuminating test 
of any method of solving differential equations (ordinary or partial) is to 
perturb the solution and study the local properties of the perturbed solu- 
tion. To illustrate, consider Euler's method for solving a single eq. (29). 
Suppose that a small error e is made at x and that Zj is the Euler solution 
of eq. (29) with this error at x . 

Let 

r\j = Zj - \Jj. 

Then, since yj and Zj each satisfy eq. (31), one finds, with the mean value 
theorem, that 

Vj+i = 1j ( 1 + h — (xj, Vi + OVj) ) , < 6 < 1. 
\ By / 
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Consider now a small enough neighborhood of xq so that second order 
effects may be neglected, i.e., that df/dy may be taken to be a constant d. 
Then 

Vi+i = iy(l + hd) = , (1 + hd) j+1 = e[(l + hd) llhd ]hd(j + 1). 
If attention is focused on a fixed point, 

X = Xq + (j + l)h, 

and if /i is allowed to approach zero, rjj+i approaches e exp (x — x )d. 
The error at x, due to the error e at x , thus remains finite as the grid size 
goes to zero, and the method is said to be locally stable. The error grows 
with x if df/dy is positive; otherwise it decreases. 

The same method shows that the other extrapolation formulas 2, 3, and 
4 of Table 9 cannot by themselves be used to solve differential equations 
because they are locally unstable, i.e., the error at x due to a given error 
at x becomes infinite as the grid size goes to zero. 

Solutions for Known yy+i. If yj+\ is obtained in some fashion, y'y+i 
can be found from eq. (29). Using eq. (30) for hy'j +i in addition to the 
equations used to obtain Table 9 results in Table 10. 

Table 10. Extrapolation Formulas 

For- 
mula I yj+i hy'j+i yj hy'j yj-i hy'j-i yj-2 hy'j-2 yj-z hy'j-z Error 

5 1 i 1 -5? (Trapezoidal formula) -fcV 3) /4 

611 | 3I3 (Simpson's rule) -h 5 y^/Q0 

TO 1 Ji- 2_7.27.2_7a_7 ■. 3_ 3 ,7 (7) 

I * 1 11 —11 11 11 ii 1 11 — TTiin V 

Heun's second order method has its basis in formula 5, Table 10, the 
trapezoidal formula. It is a considerable improvement on Euler's method 
since a much larger grid size may be used. It is just as stable as Euler's 
method, requires no past history, and calls for substitution in eq. (29) only 
once per point. 

One uses Euler's formula for a first value of 2/y+i, eq. (29) to find i/'y+i 
and then Heun's formula for a better value of ?//+i. It is not necessary to 
recompute y'j+i. The process may be iterated if desired. 

The procedure which Milne (Ref. 24) recommends most highly for solving 
ordinary differential equations uses 

y J+1 = Z/y_3 + 4V/-1 + M/Wj-i ~ Wj-2 + v'j-s) + |f^ (5) 
to extrapolate and formula 6, Table 10, which is Simpson's rule, 
y i+1 = yj-! + h/Wi-i + ±y'j + y' j+ i) - h 5 y^/90 
to recalculate. 
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Solution by formulas 2 and 6. A procedure which requires less past history 
and therefore is better for starting and at discontinuities uses formula 2, 
Table 9, to find a third order approximation to ?/y +1 and Simpson's rule to 
recalculate. Either procedure calculates derivatives only once per point. 

Formula 7, Table 10, is unstable and therefore useful for extrapolation 
but not for recalculation. 

Method of Adams and Bashforth (Ref. 25). This approach is best 
expressed in terms of differences: 

Vy'i = y'i ~ V'i-ii 
VVy = Vfo'y - y',-i).= y'j ~ 2 2 /'y_ 1 + ?/y_ 2 , 



Vj+i = Vi + Kv'j + Vy'y/2 + 5VVi/12 + 3VVi/8 + 251vYy/720 +...). 

For solutions whose derivatives of some order are everywhere continu- 
ous, this method has the advantage of yielding arbitrarily high order of 
approximation with only one evaluation of derivatives per point. For auto- 
matic computer use, it has several disadvantages which lead to its rare use. 
A special starting process is required; it is awkward to change grid size; at 
each isolated discontinuity, the special starting process must be used again. 

The Runge-Kutta Method. Like Euler's and Heun's methods, this 
method avoids these difficulties (Refs. 26 and 27). It has several forms. 
One of the best known, which has a truncation error proportional to h 5 , is : 

Uj+i = Vi + (h + 2k 2 + 2/c 3 + fc 4 )/6 

fci = hf(x jt yj), 

k 2 = hf(xj + h/2, yj + h/2), 

h = Mxj + h/2, Vi + k 2 /2), 

lH = hf(xj + h, ijj + k 3 ). 

The Runge-Kutta method was recently adapted to automatic computers 
by Gill in a form which concentrates on saving memory and reducing round- 
off error (Ref. 28). 

All forms of the Runge-Kutta method have the disadvantage that the 
derivatives must be evaluated several times, four in these two cases. 

Fourth Order Method. This method has been used extensively on 
automatic computers since 1946 and has been carefully studied by Dims- 
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dale and Clippinger (Ref. 29). It consists in extrapolating for y 3 - +2 by the 
third order formula using one past point (see Table 11): 

(32a) y j+2 = Vj _ 2 + 4fo y _ 2 - Vi ) + 4h(2y'j + y'j- 2 ) + 2h% w . 

The derivative y' 3 -+2 is then found and also i/y+i by 

(326) y j+1 = ( Vj + y i+2 )/2 + (fc/Wy - 2/'y+ 2 ) - A 4 j/ y < 4 >/24. 

The derivative ?/'y+i is then found, and t/y +2 is redetermined by Simpson's 
rule: 

(32c) y j+2 = Vj + h/3(y'j + 4y' i+1 + y' j+2 ) + h 5 y/ 5) /90. 

Isolated discontinuities are made to fall at odd-numbered grid points by 
adjusting h. To start, or at points where the grid size is altered, eqs. (32c) 
and (326) are iterated, and eq. (32a) is not used. Thus, like Runge-Kutta's 
or Gill's methods, this method requires no past history, and is well suited to 
starting, discontinuities, and change of grid size. By the addition of a 
single point from past history, it achieves the efficiency of Adam's, Milne's, 
and other methods requiring only one evaluation of derivatives per point. 

Higher Derivatives. Sometimes eq. (29) can be easily differentiated. 
In this case a fourth order, stable procedure requiring no past history is 
obtained by eliminating /i 3 y (3) , and h 4 y w from eqs. (30a), (306), and (30c) 
at i = 0, 1 : 

Vj+i = VJ + Wi + 2/'y+i)/2 + h 2 (y"j - y"y+i)/12 + ^ (5) /720. 

By adding a single point from past history, one obtains the predictor, 

y j+1 = S2 Vj - 3%_i - 2h(8y'j + 7j/' y _i) 

+ /> 2 (92/" y - 4y',-_i)/2 + /*V 6) /720, 
and the seventh order corrector, 

y i+1 = yj-i + 2y } - + 3h(y' i+1 - y f j-i)/S 

+ h 2 (W'j ~ V"j-i ~ y"/+i)/24 + ^V 8) /60450, 

which can be used except at the start, at discontinuities, and at grid change 
points. 

Method of Brock and Murray. A method which takes advantage of the 
fact that differential equations are locally linear with constant coefficients 
and therefore have solutions which are locally linear combinations of expo- 
nentials has been developed by Brock and Murray (Ref. 48). 

Extrapolation to Zero Grid Size. If a method has a local error 
proportional to h n+l , it has an error at a given x proportional to h n , since 
the number of local errors made going from x to x is (x — x )/h. By call- 
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ing e the error at x, and the exact answer, y, then, 

(33) e = y -y = ah n + bh n+1 + v , 

where the remainder, 77, goes to zero as h n+2 . If eq. (29) is solved numeri- 
cally at two grid sizes, h x and h 2 , one may write eq. (33) at both grids and 
solve for y : 

(34) y = y x + (y x - y 2 )r n /(l - r n ) + bh 2 n+1 r n (l - r)/(l - r n ) 

- (i?i - r n V2 )/(l - r n ), 

where r = h 1 /h 2 . Richardson (Ref. 30), who invented this procedure, 
called it "extrapolation to zero grid size." Looking at the next to last 
term, one sees that it would be more apt to call it "increasing the order 
of accuracy from n to n -f 1-" Equation (34) is useful in many ways. For 
example: (a) One can solve (29) at two grid sizes and use eq. (34) to get a 
better answer at common points, (b) One can solve (29) at one grid size 
and occasionally take a step at two grid sizes by using the second term to 
estimate the error and adjust the grid size. (With this procedure it is im- 
portant to use methods which depend on little past history.) (c) One can 
take every step at two grids, use eq. (34) to improve the accuracy before 
proceeding, and also use the second term to adjust the grid size. 

Boundary Value Problems or Distributed Conditions. It may 
happen that not all components of y are specified at one value of x. In- 
stead, some of the components of y may be given in terms of the others at 
two or more points. 

Approach A. Perhaps the most obvious approach to this problem is to: 

1. Assume initial conditions at x . 

2. Solve the problem. 

3. Assume other initial conditions. 

4. Resolve the problem. 

5. Interpolate between the initial conditions for initial conditions which 
will satisfy one of the other given conditions at some other point. (This 
is based on the theorem that the solutions of differential equations are, 
under suitable conditions, continuous functions of their values at particu- 
lar points.) 

6. Reiterate this process until all conditions are satisfied. If there are 
many conditions to be satisfied by varying the same number of compo- 
nents of y at Xq as parameters, the interpolation process becomes quite 
complicated. If convergence is also slow, it may be necessary to solve the 
differential equation thousands of times, treating the different equations 
and distributed conditions as simultaneous equations for all the variables 
at all the points. 
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Approach B is to consider all the distributed conditions and the approxi- 
mating equations simultaneously. For instance, consider the second order 
system : 

(35) y' = f(x, y, z), z' = g(x, y, z), 
with the distributed conditions: 

(36) y(a) = A, ky(b) + lz(b) + my'{b) + z'(b) = 0. 
One might use Heun's approximating difference equations: 

^■+! ~ yj = (f( x J> yj> z ^> + f( x J+i> yj+i> zy+i))(V2), 

(37) 

zy+i - Zj = (g(xj, y j} Z-) + g(x j+l , y j+1 , z j+l ))(h/2). 

Replacing a, b by x , x n one would write the side conditions eq. (36) in the 
form 

, x yM = A, 

(38) 

m(y n - Vn-i) + (zn - Zn-i) = (V2)[w/ n _i + g n -i - ky n - lz n ], 

where the second eq. (36) is replaced by one equivalent to it to third order 
and /„ is written for f(x n , y n , z n ). 

Equations (37) and (38) are 2{n -f- 1) simultaneous equations for the 
2{n -f- 1) unknowns ijj, zj, j = 0,1,2, • • • ,n. They are not linear; however, 
h appears as a factor of the right members and the left members are linear. 
It is therefore natural and quite practical to define an iterative process, 
writing yf and 2/ for the ith. approximation to y 3 - and zy. 

yo = A, 

yj+S - vf = (V2X//- 1 + W" 1 ), 

(39) z i+1 * - z> = {h/2){gr l + Oj+i'- 1 ), 

m(yn - y n -l) + Zn - Z n -\ 

= (fc/2) Wn-i*- 1 + gn-i'' 1 ~ kyn 1 ' 1 ~ Z n l ~ X ). 

Approach C, useful if / and g are readily differentiable, is to perturb eqs. 
(35) by introducing rj = y — y, £ = z — z where y, z is some approximate 
solution : 

df df dg dg 

(40) v = /u + /f, f-r' + r^- 

dy oz dy dz 

It would be possible to use eqs. (39) to find y and z and then, by evaluat- 
ing the derivatives df/dy, etc., at y, z solve eq. (40) as linear equations for 
37, f subject to initial conditions t]( x o) = K^o) = 0. 
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Computer Storage Requirements and Number of Operations. 

The columns of Table 11 provide a guide to the use of the methods listed. 
Similar remarks apply to this table as in the concluding paragraph of Sect. 2 
relative to content and notation of Table 5 in that section. 

The presence of past history requirements is an important consideration 
for digital computers because this generally means programming special 
starting programs for use at boundary points, at points of discontinuity of 
the solution, or at points where the grid size changes. For this reason 
formulas requiring no past history are in general easiest to program. 

It is important in evaluating a digital computer procedure to be able to 
estimate the number of operations, multiplication times, or other index of 
the computing time. However, in practical problems in differential equa- 
tions, this time is almost completely dominated by the time to compute 
the derivative f(x, y) in eq. (29), and this, of course, cannot be determined 
except in the context of a specific problem. The next best guide to the 
volume of computations is the number of times the derivative must be 
computed per integration step, and this is listed in the last column of 
Table 11. 



Table 11. C< 


3MPUTER RE( 


3UIREMENTS IN SOL 


UTION of 


Ordinary 




Differential Equations 










Past 




Derivative 




Order 


History 


Computer 


Evaluated, 


Method 


of Error 


Required 


Storage ° 


Times/Step 


Extrapolation Formulas 










(Tables 9, 10) 










Formula 1 (Euler) 


h 2 


None 


2n + w 


1 


Formula 5 (Heun) 


h* 


None 


Zn + w 


1 


Adams-Bashforth 


Arbitrary 


k points, where n 
k is arbitrary 


(.k + l)+w 


1 


Runge Kutta 


A 5 


None 


An + w 


4 


Gill 


h* 


None 


3n + w 


4 


Fourth Order Method 


h 5 


None 


6n + w 


2 

(1st deriv. once and 

2nd deriv. once) 


Predictor-corrector formulas 










Milne 


A 5 


3 points 


5n + w 


1 


Dimsdale-Clippinger 










(using 3 iterations) 


fc 5 


None 


Qn + w 


3 


Dimsdale-Clippinger 










(using extrapolator) 


h & 


1 point 


Qn + w 


1 


5th order predictor-7th 










order corrector 


h s 


1 point 


6n + w 


2 


Extrapolation to 2?ero grid 


Order of h n+1 , 






3n 


siz-e 


at least, if 
error of for- 
mula used is 
of order h n 









"wis dimension of vector y, and w is undetermined amount of working storage and 
program storage. 
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6. PARTIAL DIFFERENTIAL EQUATIONS J. B. Diaz 

R. F. Clippinger 

Bernard Friedman 

Eugene Isaacson 

Robert Richimyer 

Introduction. A variety of physical problems, when analyzed from a 
mathematical point of view, lead to the consideration of boundary value 
problems for differential equations. In many cases, the physical quantity 
of interest is found to be represented by a function which satisfies a differ- 
ential equation in a certain domain of the independent variables. Besides 
the differential equation (which may be ordinary or partial, depending 
upon whether the independent variables are one or more than one, respec- 
tively) the "unknown" function is required to satisfy certain other condi- 
tions, which will be referred to collectively as boundary conditions. Gen- 
erally speaking, these additional boundary conditions select, from the total- 
ity of the solutions of the differential equation in question, the solutions 
which correspond to the actual physical situation under study. Example. 
The determination of the steady-state temperature in a plane circular plate 
of unit radius, whose periphery is maintained at a given temperature, 
amounts to the determination of a real-valued function u{x, y) satisfying 
the partial differential equation 

d 2 u/dx 2 + d 2 u/dy 2 = for ^ x 2 + y 2 < 1, 

and the boundary condition 

u{x, y) = f{x, y) for x 2 + y 2 = 1, 

where / is a prescribed function. (/ is essentially the preassigned tempera- 
ture distribution on the periphery.) 

An equation involving a function of two or more variables and its partial 
derivatives is called a 'partial differential equation. The order of a partial 
differential equation is the order of the highest order derivative which actu- 
ally appears in it. A partial differential equation is linear, if it is of the 
first degree when considered as a polynomial in the unknown function and 
its partial derivatives (otherwise the equation is called nonlinear). Exam- 
ple. The equation d 2 u/dx 2 + d 2 u/dy 2 = is a linear second order equa- 
tion, while the equation (du/dx) 2 + u = is a nonlinear first order equa- 
tion. 
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This section will consider linear and second order partial differential 
equations starting with some mathematical background and leading to a 
discussion of numerical methods suitable for digital computer use. The 
section will conclude with a summary table giving some significant attri- 
butes of the methods listed from the point of view of digital computer 
solution (see Table 12). 



First Order Partial Differential 


Equations 


Consider 








(41) 


F(x, y, 

dz 
dx 


2, v, q) = 


-o, 

dz 
by 



a partial differential equation of first order for sasa function of x, y. The 
general solution of this problem depends on an arbitrary function. La- 
grange showed that the general solution could be deduced from a "com- 
plete" solution, i.e., a two-parameter family of particular solutions. La- 
grange and Charpit also showed that such a complete solution could be 
deduced from the solution of the system of ordinary equations for x, y, z, 
p, q in terms of a parameter: 

dx dF(x, y, z, p, q) 

dt dp 

dF 
y> = Q = —, 
dq 



(42) z' = Pp + Qq, 

/dF dF\ 

\dx dz/ 

/dF dF\ 

\d?/ dz / 



,dy 

Cauchy showed that any particular solution of eqs. (41) is composed of 
curves he called characteristics obtained by integrating eqs. (42). 
Let 

(43) x = /(s), y = g(s), z = h(s) 

be the parametric equations of a curve through which a particular solution 
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of eqs. (41) is to be found. Then po(s) and q Q (s) must satisfy the differen- 
tial equation, 

(44) F(f, g, h, poOO, ?o(«)) = 0, 
and the condition 

(45) fVo(s) + gqo(s)'-h = 0. 

The solution of eqs. (42) subject to initial conditions (43), (44), and (45) 
can be represented by 

y = y(u, t), 

z = z(u, t), 
V = P(u, t), 
q = q{u, t). 

Thus the problem of finding the solution of eqs. (41) passing through 
curve (43) is reduced to the solution of ordinary equations which can be 
done by the methods of Sect. 5. 

To illustrate, consider the linear equation, 

x+y+z+p+q=0 

subject to the conditions 

y = z = 0, 
when 

(46) O^gl. 

When y is zero and x is outside the range (46), z is not defined. 
Cauchy's method yields the solution 

x = s + t, 

y = t, 

z = -2t + (s - 2)(e - ' - 1), 

V = e~ l - 1, 

q= -l+(i-s)r'. 

g 8 g 1. 

Eliminating s and t yields 

z = -2y + (x - y - 2)(-l + e~ y ). 
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Cauchy's method shows that, in general, if z is given along some arc of 
curve C terminated at points A and B, then z is determined in a strip 
bounded by a characteristic through A and a characteristic through B. 
Call this strip the region of determinacy, in our example, the strip between 
y = x and y = x — 1. Any method other than Cauchy's must therefore deter- 
mine these characteristics one way or another to determine the region of deter- 
minacy. 

Practically, it may be difficult to obtain dF/dx, dF/dy, and dF/dz. In 
that case, it is not possible to obtain p and q by using the last two equations 
(42). As an alternative, let the characteristic curves s = constant and the 
curves t = constant be used as a curvilinear coordinate network. The 
transformation from Cartesian coordinates x, y to coordinates s, t is gov- 
erned by the relations 

t x = y s /A, 

ty = X s f A, 

(47) s x = -yt/A, 

s y = x t /A, 

A = x t y s - y t x s , 
where t x is dt/dx, etc. 

By definition and eqs. (47), 

p = dz/dx = (z t y s - z s y t )/A, 
(48) 

q = dz/dy = ( — z t x s + z s x t )/A. 

Start along t = and use Euler's method and the first three of eqs. (42) 
to get x, y, z at each point on t = h. Use numerical differentiation to ob- 
tain x s , y s , and z s at each point on t = h. Use eqs. (48) to get p, q on the 
same carve. If more accuracy is desired, Heun's method may now be used 
to obtain better values on t = h. The same process may now be repeated 
for t = —h, 2h, —2h, etc. 

Second Order Partial Differential Equations 

The general linear partial differential equation of the second order in the 
two independent real variables x and y is 

d 2 u d 2 u d 2 u du du 

(49) a— - + 2b + c— - + d— + e— + fu = g, 

dx dx by by bx by 

where the letters a, b, • • • , g denote real- valued functions of x and y. The 
equation is called homogeneous if the "nonhomogeneous term" g is identi- 
cally zero. The linear homogeneous equation has the property that the 
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"superposition principle of solutions" holds, i.e., that if u and v are solu- 
tions, any linear combination Au + Bv, with constant coefficients A and 
B, is also a solution. 

Classification. The partial differential eq. (49) can be reduced to 
certain typical, or canonical forms by means of a suitable change of var- 
iables : 

(50) £ = &,y), V = v(x,y). 
Consider first the equation with constant coefficients 

d 2 u d 2 u d 2 u 

(51) A — + 2£— -+C— = 0, 

dx dx dy dy 

and make the change of variables 

(52) £ = ax + 0y, i\ = yx + by, 

with a, /3, y, 5 real constants. In the new variables £, rj, one has 

(53) (Aa 2 4- 2£«/3 4- C/3 2 ) -£ + (Ay 2 + 2£ 7 S + C5 2 ) -£ 

d£ 2 &r 

a 2 w 

+ 2(A«t + 5[o:5 + /3 7 ] + C/35) = 0. 

Since eq. (51) is assumed to be of second order, not all three real constants 
A, B, C are zero, i.e., A 2 + B 2 + C 2 > 0. It will now be supposed further 
that i^O. There is no loss of generality, since if A = and C = 0, too, 
then fi^O, and the equation is already in "canonical" form (see eq. 54 
below); whereas if A = and C ^ one has merely to interchange the 
roles of x and y. The classification into three types is as follows : 

B 2 — AC > 0, hyperbolic type, 

B 2 — AC < 0, elliptic type, 

B 2 — AC = 0, parabolic type. 

(The reason for the designations elliptic, hyperbolic, and parabolic is obvi- 
ous from analytic geometry, the reduction of a quadratic bilinear form 
Ax 2 + 2Bxy + Cy 2 to a sum of squares.) 

Hyperbolic Case. When B 2 — AC > 0, by choosing 0=5=1, 
-5 + V# 2 - AC -B - VB 2 - AC 

a — , y — 
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in eqs. (52) and (53), and by dividing (53) by a nonzero constant, one ob- 
tains the canonical form 

(54) 

whereas by choosing 
-C 



d 2 u 


B 


0; 



s/B 2 -AC VB 2 - AC 

one obtains similarly the canonical form 

d 2 u d 2 u 

Elliptic Case. When B 2 — AC < 0, by choosing 
-C B 



7 = 0, 5 = 1, 



7 = 0, 5=1, 



■s/AC - b 2 Vac - B 2 

one obtains the canonical form 

d 2 u d 2 u 

Parabolic Case. When B 2 — AC = 0, by choosing /3 =l,a^ — B/A 
and 5 = 1, 7 = —B/A, one obtains the canonical form 

d 2 u 

In the general case of an eq. (49) with variable coefficients, it is said to 
be of elliptic, hyperbolic, or parabolic type at a given point (x , y ) according 
to whether b 2 (x 0) y ) — a(x , yo)c(x , 2/o) is < 0, > 0, or = 0, respectively. 
If the coefficients a, • • • , g are sufficiently smooth in a neighborhood of 
(x , y ), and eq. (49) is elliptic at each point of the neighborhood, there is a 
sufficiently small subneighborhood of (x , yo) in which one can introduce 
new variables by means of eq. (50) (not necessarily a linear change of 
variables as in the case eq. (51) of constant coefficients) so that eq. (49) be- 
comes, in this subneighborhood, 

d 2 u d 2 u / du du \ 

— - -| + I Linear terms in — . — , and u J = 0. 

d£ 2 dr) 2 \ d£ d n / 

A similar statement applies in the hyperbolic and parabolic cases. 
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Of course, eq. (49) with variable coefficients may be of different type at 
different points of a domain, i.e., it may be of "mixed" type, as occurs in 
the linearized equation for the potential function of a two-dimensional 
compressible flow. Example. The equation yd 2 u/dy 2 + d 2 u/dx 2 = is 
elliptic for y > 0, parabolic for y = 0, and hyperbolic for y < 0. 

Representative equations commonly studied are: 

(a) Elliptic Laplace d 2 u/dx 2 + d 2 u/dy 2 = 

(6) Hyperbolic Vibrating string d 2 u/dx 2 — d 2 u/dy 2 = 

(c) Parabolic Heat ' d 2 u/dx 2 — du/dy = 

The variable t is usually written instead of the variable y in the last two 
equations. For a more detailed discussion of the canonical forms of eq. 
(49), as well as for the classification into canonical forms of higher order 
equations and systems of equations see Refs. 36-38. 

Difference Equations. In numerical investigations it is often necessary 
to replace the partial differential equation occurring in a given boundary 
value problem by a suitable equation involving differences rather than 
derivatives of the unknown function (see Chap. 4). The basic principle 
usually employed is none other than the fact that any partial derivative is 
the limit of a certain difference quotient. For a function of one variable, 
f(x), the difference quotients in the plus x and minus x directions, f x and 
f$, are defined by 

fix 4- h) - f(x) f(x) - f(x - h) 

JxW — 7 o-Htl JX\X) — - > 

h h 

where h > 0. The second differences of f(x) are defined as the differences 
of the first differences. There are three second differences, f xx , f X x( = fxx), 
&ndf xx . The second difference/^ is the most "symmetric" of the three: 

, . v _ f(x + h)+f(x-h) -2f(x) 
h 2 

The corresponding differences for functions of several independent varia- 
bles are defined as above, upon holding fixed all the variables but one at a 
time. For example, for a function of two variables u{x, y) : 

u(x + h,y) - u{x, y) 

u x (x, y) = 

h 

and 

u{x + h,y) - 2u(x, y) + u(x - h, y) 
u xx {x, y) = — , etc. 
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Laplace equation. By taking the difference equation u xx -f u yg = as a 
"difference approximation" to Laplace's differential equation d 2 u/dx 2 -f- 
d 2 u/dy 2 = 0, one obtains the difference equation 

u(x + h, y) + u(x - h, y) + u(x, y + h) + u(x, y — h) 

(55) — = u(x, y). 

4 

Vibrating string. By taking the difference equation u xx — u yy = as a 
difference approximation to the vibrating string equation d 2 u/dx 2 — 
d 2 u/dy 2 = 0, one obtains the difference equation 

u(x + h, y) + u(x — h, y) — u(x, y -j- h) — u(x, y — h) = 0. 

Heat. In the case of the heat equation d 2 u/dx 2 — du/dy = 0, one has 
the alternative difference equations u xx — u y = and u xx — u y = 0. 

Exactly the same procedure is applicable to first and to higher order 
partial differential equations, as well as to systems of equations. An alter- 
native approach to the numerical treatment of first order partial differen- 
tial equations can be based on the fact demonstrated in the previous sub- 
section that the solution of a first order partial differential equation and 
the solution of the characteristic system of ordinary differential eqs. (42) 
corresponding to the given first order partial differential equation are 
equivalent tasks. 

Note. The following three subsections represent results obtained at the Insti- 
tute of Mathematical Sciences, New York University, under the sponsorship of 
the United States Atomic Energy Commission Contract AT(30-1)1480. Repro- 
duction in whole or in part permitted for any purpose of the United States Govern- 
ment. 

Elliptic Partial Differential Equations 

Consider a partial differential equation of second order for a function u 
of n variables Xi, x 2 , • • •, x n . One writes the equation as follows.* 

d 2 u _ du 

(56) Lu = 2_j an hL & i \- cu = f. 

The coefficients a^, bi, c are assumed to be constant. This equation is 
called elliptic in a region R if the quadratic form 

(57) Haatej 

is non-negative definite for all values of the & such that (£i, £ 2 , • • •, In) is 
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a point in R. A typical example of an elliptic difference equation is Pois- 

son's equation, that is, 

_ d 2 u 
Aw^E =/. 

i OX{ uX{ 

Note that the condition (57) for ellipticity depends only on the highest 
order derivative terms of eq. (56). 

Dirichlet and Neumann Problems. A typical problem involving 
elliptic differential equations is one that requires the solution of a boundary 
value problem. For example, a typical problem would be to solve 

Lu = f 

in a region R given that u is a prescribed function u on the boundary B of 

the region R. Such a problem is called a Dirichlet 'problem for eq. (56). If, 

instead of the values of u, the values of du/dv, the normal derivative of u, 

are prescribed on B, the problem is called a Neumann problem. A more 

general problem is that in which eq. (56) has to be solved, given that the 

values of 

du 

hiu + h 2 — 

dv 

are prescribed on B. Here h x and h 2 are known functions. If h 2 = 0, it is 
a Dirichlet problem; if hi = 0, a Neumann problem; if neither is identi- 
cally zero, it is a mixed problem. 

Choice of Method. The standard procedure for solving a partial 
differential equation numerically is to place a rectangular mesh on R, to 
replace the differential equation at each mesh point by a finite difference 
approximation, and thus obtain a set of linear equations. The main diffi- 
culty in this procedure occurs in the process of solving the set of linear 
equations. Inverting the matrix of this set of linear equations is usually 
not convenient because the matrix is generally ill conditioned. A "march- 
ing" process such as that used for solving hyperbolic equations by which 
the values in the lines of the mesh are determined in succession from the 
values on the preceding lines is not feasible because the values on any line 
depend on the values of two preceding lines and the boundary data are not 
sufficient to determine the values on two successive lines. Because of these 
considerations, the method most frequently used for solving the linear 
equations is an iteration or relaxation method. 

Iteration Procedure. In order to discuss iteration methods, some 
notation is needed. Suppose one wishes to solve a system of equations in 
p unknowns. Let x denote a p-dimensional vector whose components are 
the p unknowns X\, x 2 , • • -, x p ; let the p X p matrix, K, of the coefficients 
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of the unknowns be nonsingular, and let & be a vector whose components 
are the p nonhomogeneous terms in the set of equations. Then write the 
system of equations as follows : 

(58) Kx = &. 
To solve this system by iteration, put 

K = N - P, 

where N and P are any matrices whose difference is K, and write (58) as 

Nx = Px + &. 

Make an estimate of the value of x and call it x (0) . A better estimate is 
possibly given by the vector # (1) such that 

Nx (1) = Px m + b. 

This process can be continued indefinitely. Define the vector a: (n+1) 
(n = 0, 1, 2, • • •) as the solution of 

(59) Nx (n+1) = Px (n) + 6, n = 0, 1, 2, • • •, 

and hope that the sequence of vectors x (n) converges in the limit to the 
desired vector x. 

The iteration method defined here is completely general in that the split- 
ting of the matrix K into two matrices N and P was arbitrary. Each dis- 
tinct split gives a different iteration procedure. There are, however, two 
restrictions on the ways of splitting K. 

(a) To find .r (n+1) from eq. (59) more easily than to find x from (58), N 
must be a matrix with an easily found inverse. For example, N might be a 
diagonal matrix or a lower triangular matrix. 

(6) For the iteration scheme to converge, it is required that all the eigen- 
values of the matrix N~ 1 P be in absolute value less than 1. It can be 
shown that this is a necessary and sufficient condition for the sequence z (n) 
to converge to x, no matter what the original guess x (0) is. 

Richardson and Liebmann Iteration Methods. The ideas of the 
preceding section will be illustrated by applying them to the solution of 
Poisson's equation 

d 2 u d 2 u 

(60) l? + ^ = f(x ' v) 

inside the unit square when 0^x^l,0^y^l. Assume that the val- 
ues of u(x, y) are given on the boundary of the square. 
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Put a square mesh of width \/p over the unit square and let 



(i A 
\p p/ 

(v-W«, 

\p v' 



/(-•-) =fa, hj = 0, 1, •••, p. 

\p p! 

By the use of the well-known finite difference approximation for the Lapla- 
cian (see eq. 55), eq. (60) becomes 

(61) Wf_i,j + Wf,y_i — 4:Uij + Ui + ij 

+ Uij+i = -g/iy, *j i = 1, 2, • • •, p - 1. 

Since the values of u 0j ; u P j (j = 0, 1, • • •, p) and of m j0 , w t - p (i = 0, 1, • • •, p) 
are given, (61) is a system of (p — l) 2 equations for the (p — l) 2 unknowns 
mj (i,j = 1, 2, •••, p - 1). 

In Richardson's method for solving eq. (61), the following iteration scheme 
is used : 



(62) 4:Ui/ n+1) = Ui_ u {n) + u lV _! (w) + w,-+i,i 



(n) 



1 

+ t*, v+ i (B) --2^' 1 = 0,1,2, •-.. 
7> 

The values of w,;/ 0) are of course the initial guess to the solution of eq. (60). 
If this method is compared with that in eq. (59), it is apparent that the 
split is such that iV is a diagonal matrix. 

One disadvantage of Richardson's method when an electronic computer 
is used is that all the previous values of it;/ n) must be stored until all the 
new values of w# (n+1) are found. This disadvantage is avoided in Lieb- 
mann's method where the new value of w*/ n+1) is calculated by using as 
many new values as are available. Thus, if the values of ua are calculated 
in order along each row from left to right and the rows in order from bot- 
tom to top, the following iteration scheme would be used: 

(63) 4u,-,y< B+1 > = t*i_i./ n+1) + «,v-i (w+1) + "*+i.y (B> + «y+i (B) - -Ja- 

V 

It can be proved that the method defined by eq. (63) would converge twice 
as fast as that defined by eq. (62). 

The rate of convergence can be still further improved by using an extrap- 
olation parameter a, thus obtaining what is called Liebmann's extrapolated 
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method. The iteration scheme is now this: 
4m# (b+1) = 4(1 - a)Uij {n) 

+ a u,-_ w (n+1> + t*,vy_i (n+1) + t*W B) + M*,y + i (B) - -Jij 
L p J . 

The value of a for which convergence is fastest is found by solving the equa- 
tion 

a V - 4a + 4 = 0, 

where t m is the largest eigenvalue of the Richardson scheme eq. (62). For 
the case considered, t m = cos (ir/p). For a rectangular mesh with p divi- 
sions in one direction and q divisions in the other, 

t m = i[cos (tt/p) + cos (v/q)]. 

In general the use of the extrapolated Liebmann method with the best 
value of a will be much faster than the unextrapolated Liebmann method. 
Line Iteration Schemes. Another iteration method which is useful 
in many cases is given by the following scheme: 

(64) 4w ii (B+1) - M W _i (n+1 > - M f .y+i (n+1) 

V 

In this scheme instead of solving for the values of u at a point ij, solve 
for all values of u on the ith. column in terms of the values of u on the 
(i — l)-th and (i + l)-th column. That is why eq. (64) has been written 
with the left-hand side containing all the w-values on the ith column. Since 
at each step the value of the right-hand side is known for all values of j, 
the three-term relation defined by eq. (64) is solved for the values of u on 
the t'th column. (The method of solving the three-term relation is ex- 
plained in the subsection on Hyperbolic Partial Differential Equations.) 

Instead of solving for the values of it on a column, one may solve for the 
values of m on a row. In that case use the following scheme: 

(65) 4w iy (B+1) - M 1 -+i,/ n+1) - Wf_i./ n+1) 

= V ■ ■ , ( n +!) _L 7 j . . , , («) .{.. 

V 

Again this three-term recurrence scheme is solved for the values of u on the 
jth row starting with j =1. 

The Method of Peaceman and Rachford (Ref. 35). This seems to 
be one of the quickest iterative methods for solving an elliptic differential 
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equation. It is essentially a line iteration scheme which uses columns and 
rows alternately. The explicit description of the method is contained in 
the following formulas: 



= -Ww_l (2n) + (2 - p»W 2 »> - U ilj+1 ^ + -J {i ; 



v 2 ' 



u;_i,/ 2n+1) - (2 + PnW 2n+1) + uW 2 " +1) 
= -Ww_i (2n) + (2 - Pn ) 

Ui,U 2n+2) ~ (2 + PnW 2n+2) + ^,y + i (2w+2) 

= -<W 2n+1) + (2 - PnW 2w+1) - tW 2n+1) +4/* 

v 

Here p n is an extrapolation parameter which is to be determined so that the 
method will converge as quickly as possible. In the present case Peaceman 
and Rachford suggest putting p n = p& if n = k (mod p), where 



p k = 4 sin 



2 (2k + l)r 



4p 



Variational Principle. An important characteristic of elliptic dif- 
ferential equations is that they can be obtained as the Euler equations of 
problems in the calculus of variations. Physically, this implies that the 
problem possesses an energy integral whose minimum value is given by the 
solution of the elliptic partial differential equation. For example, in Di- 
richlet's problem the integral 

(66) ff(Vu) 2 dxdy 



must be a minimum in the domain of all functions u satisfying the preas- 
signed boundary conditions. In problems with mixed boundary conditions 
the integral (66) must be modified (for details see Ref. 37). As another 
example, in elasticity problems involving plates, the integral 

(67) f f(Aw) 2 dx dy 

must be a minimum in the domain of all functions u satisfying the pre- 
assigned boundary conditions. 

For numerical purposes the energy integral can be approximated by a 
sum involving the values of the unknown function u at a set of points in- 
side the region R. Then choose values of u so that the sum will be a mini- 
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mum. For example, (66) would be approximated by 

(68) 2 2 [( U H - Ui-u) 2 + fay - Mij_i) 2 ], 

i j 

and (67) by 

(69) X) Z) Kv '- i(**i./+i + w w-i + u i+u + w,-_i,,-)] 2 . 

* i 

As these illustrations show, the sum is a quadratic form in the values w z y. 
By differentiation with respect to u { j a set of linear equations is obtained 
whose solution will make the sum a minimum. Simple algebra shows that 
when this method is applied to eqs. (68) and (69) the standard difference 
equations for Laplace's equation or the biharmonic equation are obtained. 
Any iteration method which at each step reduces the value of the sum 
must automatically converge to a minimum value. Use of this fact easily 
shows that the various schemes proposed above do converge to a solution. 
The variational principle is also useful in determining how the boundary 
conditions should be taken into account. 

Hyperbolic Partial Differential Equations 

The equation of the vibrating siring will be used to illustrate some finite 
difference methods for solving problems involving a second order hyper- 
bolic partial differential equation. 

If the end points of the string are held fixed at x = and x = 1, the 
deflection of the string u(x, t) is determined from the initial deflection 
u(x, 0) = f(x), and the initial velocity u t (x, 0) = g(x). 

The conditions describing the motion are: 

1 
P.D.E. — u tt = u xx , for < x < 1, t > 0. 

(70) I.C. u(x, 0) = f(x) 

u t (x, 0) = g(x), for < x < 1, t = 0. 
B.C. u(0, t) = w(l, t) = 0, for t ^ 0. 

(Note the abbreviations P.D.E. for partial differential equation, I.C. for 
initial conditions, B.C. for boundary conditions and c 2 for the ratio of the 
string tension to the string mass per unit length.) 

The finite difference methods are based on the replacement of derivatives 
by difference quotients in a rectangular lattice over the (x, £)-plane. The 
mesh points of the lattice are (x{, tj), where X{ = ih, tj = jk, with i and j 
taking on integer values. Let the x interval be such that Nh = 1 with 
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some integer N, and let k/h = X be the ratio of mesh widths. Figure 1 
schematically indicates the lattice work with N = 6, X = 1. 



« = 



u = 



u=f 



Fig. 1. Mesh points of lattice for N = 6, X = 1. 

Explicit Finite Difference Scheme. The simplest replacement of 
eq. (70) by finite difference equations is found by using central differences 
for derivatives as follows: 



P.D.E. 



U(x it t j+ i) - 2U(xi, tj) + U(x it tj_x) 



27,2 



c z k 



U(x i+1 , tj) - 2U(x{, tj) + U(xi- U tj) 



h 2 



I.C. 



U(xi, 0) = f(xi) 



for 1 ^ i£ N - 1, j ^ 1. 



k 2 c 2 
~2~h 2 



U(x h k) = f(xi) -f- fcpfo) + — [/(x,- + i) - 2/(x,-) +/(aj t -_i)], 



for 1 ^ i < N - 1. 



B.C. Z7(0, < y ) = C/(iV, « y ) = 0, for j ^ 0. 



The initial condition which determines U at the first line t = k is derived 

from Taylor's expansion 

A; 2 
u(x, k) = w(a:, 0) + ku t (x, 0) + •— u tt (x, 0) 



together with the relation 



Uu(x, 0) = c 2 w xx (a;, 0) = c 2 



eta 2 " 
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With the notation / t - = f(x{), Ui t j = U(x{, if), the equation may be re- 
written to exhibit the explicit manner in which the solution can be calcu- 
lated. 

P.D.E. U iti+1 = c 2 X 2 U i+ltj + (2 - 2c 2 X 2 )U itj . 

+ c 2 X 2 £/;_ w - Uij-u iorj^l,l£i£N- 1, 

Ui.o = fi 



(71) I.C. 



B.C. 



c 2 X 2 c 2 X 2 

Ui.i = —fi+i + (1 - c 2 X 2 )/, + —fi-i + kg it 

k 
Uo,j = Un,j = 0, with X = -• 



Convergence of Finite Difference Scheme. It is not clear a priori, 
that the solution U of (71) will converge to the solution u of eq. (70) as 
h, k '-> 0. 

In fact if as h, k — > 0, X is held fixed, and such that X > 1/c, the solution 
[/ will not converge to w for all initial displacements. This may be verified 
by noting that the solution of the initial value problem for the infinite 
string is given by 



(72) 



-^ ~x+ct 

u(x, t) = i[f(x + ct) + f{x - ct)] + - I <s 

2C J x—ct 



flftt) * 



Formula (72) shows that the solution u(x, t) depends solely on the initial 
data in the interval (x — ct, x + ct). The solution U(x, t) depends solely 
on the initial data in the interval [x — (t/X), x + (t/X)]. Hence if X > 1/c, 
it is possible to vary / and g in the intervals [x — ct, x — (t/X)] and 
[x + (t/X), x + ct] in such a way that the solution u(x, t) is changed, but 
yet U(x, t) is unaffected. Hence if X > 1/c, the solution U(x, t) cannot 
converge as h, k — > since it would have to converge to different values. 

Hence it is necessary for convergence that X ^ 1/c, i.e., the "domain of 
dependence" for the solution of the finite difference equation should con- 
tain the domain of dependence of the solution of the differential equation. 
In fact, if X ^ 1/c, U does converge to u as h, k — » 0. The proof of con- 
vergence may be made to rest upon the Fourier series representations of 
the solutions of eqs. (71) and (70), namely, 



U(x, t) = 23 (An cos nJ + B n sin fi n t) sin 



nx. 



where /x n is determined from the condition sin n n k/2 = Xc sin nh/2; and 

00 

u(x, t) = 53 ( a n cos nc t + &n sin net) sin nx. 

n=l 
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Roundoff and Truncation Errors. The calculation of U is in practice 
effected by rounding to a finite number of decimal places. The equations 
which determine U are 



Ui,o — fi + Ru 



0) 

c 2 X 2 c 2 X 2 

Ui, x = — A-+i + (1 - c 2 X 2 )/; + -—fi-i + kg { + R itl , 

U ilj+1 = c 2 \ 2 U i+llj + (2 - 2c 2 \ 2 )Ui,j + c 2 \ 2 Ui_ u - Uu-i + Rij+i, 
Uo,j = U]v,j = 0, 

where R p>q is the roundoff error. The truncation error T PtQ is defined by 
substituting u into eq. (71) as follows: 

u i,0 = fi "T J- i,0) 

c 2 \ 2 c 2 X 2 

Ui.i = —fi+i + (1 - c 2 X 2 )fi + —fi-i + kg* + r t - tl , 

Mi,y+1 = C 2 \ 2 U i+ i,j + (2 — 2c 2 \ 2 )u it j + C 2 \ 2 Ui__i,j — Ui,j-1 + r.-.y+i. 

It is easily verified that T ii0 = 0, T itl = 0(k 3 ), T itj = 0(/c 4 ) where 0(k n ) 
represents a quantity which is bounded in absolute value for all sufficiently 
small k by Mk n with some constant M. 

It is reasonable to require that the roundoff error be of the same order 
of magnitude as the truncation error or smaller, in order that the number 
of digits carried in the calculation be appropriate for the interval size. 
With this restriction, the total error e^y is 0(T 2 k 2 ) for any finite time, T, 
where 

6i,j = U{j — XJij and ^ j ^ T/k. 

Implicit Schemes. The restriction k/h = X ^ 1/c may be relaxed by 
using an implicit scheme. That is, it is possible to take larger time steps 
at the expense of more involved calculations as follows : 



Ui,j+i — 2Ui,j + U{,j_i c 
~k 2 = tf 



P.D.E. - 2 = — 2 [a 2 (U i+ltj+1 - 2U ilj+1 



+ Ui_u+x) + (1 - 2a 2 )(U i+ltj - 2U itj + Ui_ U ) 
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The equations when solved for the unknowns at the (j + l)-st time step 
have the form 

(73) (ac\) 2 U i+uj+l - [1 + 2{ack) 2 ]U w 

+ (acX) 2 C/,_ u+1 = W, for i = 1, 2, • • •, iV - 1, 

where W involves information on the two preceding lines (j and j — 1). 
The labor involved in solving eq. (73) is minimal since the (N — 1) X 
(JV — 1) matrix of coefficients is in triple diagonal form. At the same time 
the condition on X which insures convergence of U to u as h, k — > is 

XV ^ — -. for ^ a 2 < i, 

1 - 4a 2 4 

and no restriction for 34 ^ ab- 
solution of Triple Diagonal Systems. The equations 

&1.-C! + C X X 2 +0 + • • • 

a 2 ^i + b 2 x 2 + 02^3 +0 + • • • 

+ a 3 x 2 + 63^3 + C3Z4 + + • • • 

+ • • • +0 -j- a N -xX N - 2 + &Ar_ia;Ar_i + cat-iXat = t/jv—i 

+ • • • +0 + ajvzjv_i -f b N x N = y N 

may be solved by eliminating the unknowns in succession from the equa- 
tions. By starting at the top the system can be put in the form 



+ 


= V\ 


+ 


= v% 


+ 


= 2/3 



xi + C x x 2 + + • • • 


+ 


= Y, 


+ x 2 + C 2 x 3 + • ■ 


■• +0 


= Y 2 



+ • • • +0 + x N _! + C N _ x x N = Y N _ X 
+ • • ■ +0 + x N =Y N 

The numbers Ck and F^ may be recursively computed from the formulas 



(74) 



c i 2/i 

d = - ; Y x = — • 

„ ^ Vk- ukYr-x 

bx — (IrCr—i bx — (IkCk—i 

for K = 2, 3, • • ■ , iV. 
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It is now easy to solve for the xk beginning with xn as follows: 

%n = Y N , 
(75) 

x K = -Y K - C K x K + x , for K = N - 1, N - 2, • • •, 1. 

The Von Neumann Criterion for Convergence. A quick method 
for heuristically testing the convergence of a finite difference method has 
been attributed to von Neumann. 

In the case of linear differential equations with variable coefficients, the 
method consists in replacing the coefficients by constants and then finding 
all solutions of the difference equation of the form 

U(x, t) = e^e^ x , with /3 real. 

If | e 7 ' | ^ 1 for t ^ 0, for all 13 and for all admissible values of the coeffi- 
cients, the finite difference method is said to be stable, otherwise not. 

The von Neumann "test" for convergence is the same as for stability. 
In practice, this test for convergence is as simple as any a priori calculation 
could be ; in addition, it has been shown to be a sufficient condition for con- 
vergence for a large number of cases. 

Parabolic Partial Differential Equations 

Finite difference methods for parabolic equations are similar to those for 
hyperbolic equations. The present discussion will be restricted to equa- 
tions of the first order in time and second order in one or more variables. 
Illustrative methods will be given for: 

(a) The linear heat flow equation in one dimension. 

(b) A quasilinear equation in one space variable and time. 

(c) A linear parabolic equation in two space variables and time. 

For diffusion or heat flow in one dimension there is an initial value prob- 
lem consisting of a partial differential equation (P.D.E.), an initial condi- 
tion (I.C.), and boundary conditions (B.C.) for a function u(x, t). In the 
simplest case, these are : 

1 
P.D.E. -u t = u xx , for < x < 1, t > 

(76) 

I.C. u(x, 0) = f(x), for < x < 1, t = 

B.C. u(0, t) = w(l, t) = 0, for t ^ 0, 

where a is a positive constant. 

Just as for the vibrating string problem one introduces a lattice of net 
points (x{, t/) in the (x, £)-plane, with X{ = ih, tj = jk, where i and ,; are inte- 
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gers, h and k are increments of distance and time, respectively, and h is 
such that Nh = 1 for some integer N. By using the notation U{j = 
U(x{, tj) as before, where U(x, t) is the approximation to u(x, t) to be ob- 
tained from the numerical calculation, the system (76) is replaced by the 
difference equations 



P.D.E. 



Ui,j+i — U{,j Ui+i,j — 2Ui,j -\- Ui—\,j 



ak h 2 

(77) for 1 ^ i ^ N - l,j ^ 1 
I.C U it0 = f(xi), for 1 ^ i ^ N - 1 

B.C. U ,j = U Nlj = 0, fori ^ 0. 

These equations are explicit and their solution converges to the solution of the 
problem (76) as the mesh is refined, provided that h, k — > in such a way that 

(78) ^ = Constant ^ J. 

Convergence. By convergence is meant the following. One considers 
a sequence of calculations with progressively finer meshes and one supposes 
that x, t is some fixed point common to the lattices of infinitely many of 
these calculations. Then U(x, t) — > u(x, t) as h, k —> under the above 
restrictions. This convergence occurs if the initial function f(x) is any 
function continuous in the interval ^ x fS 1. If the condition (78) is 
violated by taking ak/h 2 = constant > ^ as h, k — » 0, the solution of the 
difference eqs. (77) diverges for all but special cases in which the initial 
function f(x) has a terminating Fourier series. One says that the equations 
are unstable under these circumstances and that (78) is a condition for 
stability. For general discussions of convergence and stability, see Refs. 39, 
40, and 43. 

The convergence as k — » is slower, at least in a formal sense, than it 
is for the hyperbolic problem, and its rate depends upon the smoothness 
of the initial function f(x). If condition (78) is satisfied and f(x) is analytic 
for ^ x ^ 1, the error eij of the approximation (77) is 0(Tk) for t in a 
finite interval ^ t S T. One can of course also write e it j = 0{Th 2 ) be- 
cause of the relation (78). 

The method is more accurate in the special case in which h and k are so 
chosen that 

ak 

(79) £-*. 

It is easy to verify by Taylor's series expansions that in this case there is 
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a cancellation of the first order error terms coming from the two members 
of the first eq. (77). In consequence, if f(x) is analytic, e^- = 0(Tk 2 ) = 
0(Th 4 ). 

A condition of the form (78) is perhaps not unexpected from the point 
of view of the domain of dependence of the differential equation, which is 
not confined to a small interval as it was for the vibrating string problem. 
That is, u(x, t), for any t > 0, depends on all the initial data, i.e., on the 
values of f(x) for the entire interval ^ x ^ 1 . For any finite values of h 
and k the difference equations of course possess a restricted domain of 
dependence, but as the mesh is refined this domain opens out (because eq. 
78 requires that k vary as h 2 ) so as to include all past values of the function. 

Implicit Equations. Implicit difference equations can be constructed 
for the heat flow problem in many ways. For example, one can replace the 
first eq. (77) by the equation 



t^t.y+i — U{,j 1 
~ak ~h 2 

+ (1 - a)(U i+ u ~ 2Utj + Ui-u)], 



(80) -^—^ = ~ 2 [a(U i+u+1 - 2U ilj+1 + Ui- U+1 ) 



where a is a constant. The resulting method reduces to the foregoing ex- 
plicit method for a = 0, to the so-called Crank-Nicholson method (Ref . 41) 
for a = }/% and to the method of Laasonen (Ref. 42) for a = 1. 

The condition for convergence of the solutions of eq. (80) as h, k — » is 

(81a) 



ok 1 
< 

h 2 ~ 2 - 4a 


ifOZ 


a < 


l 

2 


No restriction if § = « 


^ 1. 





(816) 

The Crank-Nicholson and Laasonen schemes, in particular, are therefore 
unconditionally stable. 

The optimum value of a from the point of view of accuracy is, surprisingly 
enough, not a = }/2, but rather 

1 h 2 



a = 



2 12ak 

(which includes eq. 79 as a special case). It is easily verified that the con- 
dition eq. (81a) for stability is satisfied in this case. 

The implicit equations resulting from the use of eq. (80) can be solved by 
the algorithm described above for the hyperbolic problem, that is, by the 
use of eqs. (74) and (75). 

Variable Coefficients and Nonlinear Problems. The procedure can 
be generalized so as to apply to problems with variable coefficients and 
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nonlinear problems. For example, the quasilinear equation 

du d 2 u du 

— = f(x, t,u)—--\- g(x, t, u) — 
dt dx dx 



(82) — = f(x, t,u) — -\- g(x, t,u) — + S(x, t, u), 



where /(re, t, u) ^ constant > for < rr < 1, < £ < T, can be replaced 
by the difference equation 

(83) Vi ' i+l " Vhi 



Ji<j 
I 2 



= ij[a(U i+1 , j+1 -2U i , j+1 +U i _ 1 , j+1 ) + (l-a)(U i+1 , j -2U i , j +U^ 1 , j )] 

-\ — — (Ui+i,j — Ui,j) + Si,j, 
h 



which was abbreviated by writing f it j for f(x{, tj, Uij), and similarly with 
respect to g and S. Suppose that the initial and boundary conditions are 
given the same treatment as before. Alternative approximations to eq. 
(82) can be obtained by replacing the coefficient of gij/h by the backward 
difference Uij — Ui—u or by expressions such as 

2(^+1./ — Ui—u) 
or 

~i(Ui+i,j + C/i+i ,y-4-i — Ui—ij — Ui—u+i), 

and in many other ways. 

If g{x, t, u) and S(x, t, u) are zero, one expects the condition for stability 
to be 

k 1 

f(x, t, u(x, t)) — < for < x < 1, < t ^ T, if < a < h. 

h 2 ~ 2 - Aa - - » - - - 2 

No restriction, if \ ^ a ^ 1, 

in analogy with the simple heat flow problem. Generally the effect of the 
low order terms on stability is merely to replace the ^ sign in the first line by 
the < sign. This change has of course no significance for machine calcula- 
tion where roundoff destroys the possibility of distinguishing between exact 
and near equality. But to be on the safe side, a conservative statement 
is given. Calculation based on eq. (83) converges as h, k — » provided that 



k 
(84) max/(£, t, u(x, t)) — ■ 

x.t.h.k h 



1 

< ifO<a<i 

2 - 4a 2 



<oo if % ^ a SI. 

No mathematically rigorous proof of convergence has been given; the 
statement made above is based on simple intuitive arguments coupled with 
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rigorous discussion of the linear cases with constant coefficients, with em- 
pirical evidence from many calculations, with Fritz John's rigorous treat- 
ment (Ref . 44) of a wide class of difference equations which includes as a 
special case eq. (83) when / and g are independent of u and a is taken equal 
to zero, and with Douglas' rigorous treatment (Ref. 45) of the case in which 
a = 1, g = 0. 

Note that in the nonlinear case one cannot guarantee stability in ad- 
vance : rather, the inequality (84) must be verified constantly as the calcu- 
lation proceeds and, if the solution at any instant threatens t6 violate the 
condition, the time interval k must be reduced. 

Parabolic Equation in Two-Space Variables. As a last example, the 
simple parabolic equation in two-space variables for a function u = u(x, y, t) 

du d 2 u d 2 u d 2 u 

(85) — = A — + 2B +C — 

dt dx 2 dxdy dy 2 

where A, B, C are constants such that 

A > 0, C> 0, AC - B 2 > 

can be approximated by an analog of the general implicit eq. (80). Because 
of the number of variables it is convenient to introduce a slightly different 
notational convention for this problem by calling the increments At, Ax, Ay 
and by relating the time variable £ to a superscript n as follows. Let 



U jd n = U{jAx,lAy,nAt), 
A 
(Ax) 



*i.i" = 7TXI (C7>+i.i" - 2tfy.i n + tfi-i.i B ) 



B 

+ ;n — 7~ (^i+i-z+i 11 — #i-i.i+i n — Uj + i,i-i n + Vj_i,i_i n ) 

2 Ax Ay 



+ TTTi (tfi.i+i" ~ 2U i>i n + U jt i-i n ). 



c 

With these abbreviations, the approximation to eq. (85) is 

tt w+1 TJ . n 

(86) — — = a^j,r +l + (1 - a)^,i n . 

At 

If a = 0, the equations are explicit. If a 5^ 0, they are implicit; in this case 
the simultaneous equations to be solved at each stage of the calculation are 
for unknowns Uj t i n+1 where, for example, j — 1, 2, • • -,J,l = 1, 2, • • -,L, 
with n fixed. These simultaneous equations are of elliptic character and 
are usually solved by one of the relaxation methods. 
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Table 12. Characteristics of Some Numerical Methods of Solution 
of Partial Differential Equations 

(w = storage requirements for computer program and working storage. Additional 
storage requirements given in terms of dimension N of lattice.) 



Elliptic ' 



dx 2 






+ ^ri = /(*, y) 



Method 
Richardson (eq. 62) 
Liebmann (eq. 63) 
Extrapolated Liebmann 

(optimum a) 
Peaceman and Rachford 



Storage 

2N 2 + w 
N 2 + w 

Same as Liebmann 
N 2 + w 



Hyperbolic: — 2 2 



ld 2 u 
c 2 



dht 
dx 2 



Evaluation 

of One Uif 1 , 

Time 6 

4r 

4t 

7r 
9r 

(c = constant) 



Method 
Explicit (eq. 71) 

Implicit (eq. 73) 



Storage 
2/V + w 

4N + w 



Stability Condition 

c 

X 2 c 2 ^ — 1 —- 2 
1 — 4a 2 

if ^ a 2 < \ 

Unconditionally stable 

.if 1 ^ a 2 



Explicit (eq. 77, 




or eq. 80 with 


N + w 


a = 0) 




Crank-Nichol- 


4N + w 


son (eq. 80 with 




« = *) 




Laasonen (eq. 


4N + w 


80 with a = 1) 




Generalized im- 




plicit method 


4IV + w 


(eq. 80) 





_, . . . . 1 du d 2 u 

Parabolic: d - — = —-5 

a at ox* 



h 2 = 2 



(c = constant) 



Unconditionally stable 

ok 1 

h 2< 2-Acl 



Total 

Calculating 

Time 6 

lliVV 

5.W 4 t 

15N 3 t 
(34iV 2 log, N)t 



Truncation 

Error for 

1 Step c 



0(A; 4 ) 



Oik') 



\°W ^ 1 
l v ' h 2 6 



Unconditionally stable 0(k) 



(Kk) 

\ Oik); Oik 2 ) 

2 12<rk 



if ^ a < - 

Unconditionally stable Oik) 
jf I ^ a ^ 1 

"(1) An N X N lattice is assumed. (2) One has the option, of course, of computing 
fij or storing a table of/,-/. In the latter case the storage is increased by iV 2 , whereas in the 

[Footnote continued on p. 14-88.J 
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The stability condition for eq. (86) is 

A At C At 1 . 

(Ax) 2 (Ay) 2 2 - 4a 2 

(87) . . 

No restriction, if \ ^ a ^ 1. 

Equation (86) can be generalized so as to apply to problems with variable 
coefficients and to quasilinear problems, as well as to problems in three or 
more space variables. 

Summary Table 

Table 12 gives some significant attributes of methods from the point of 
view of digital computer solution. 
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1. OPERATIONS RESEARCH AND MATHEMATICAL MODELS 

General Nature of Operations Research 

Development. Each branch of science has emerged as a result of a 
concentration on a particular class of problems and on the associated de- 
velopment of scientific methods, techniques, and tools for the solution of 
these problems. One such class of problems, spotlighted early in the cen- 
tury by such farsighted men as F. W. Taylor, is concerned with the manage- 
ment function. Recent concentration of scientific effort on this set of 
problems has given rise to the science called "Operations Research," often 
abbreviated to "O.R." For an extensive discussion of the general nature 
of O.R.,, its. development, structure, prototype problems, methods, tech- 
niques, and tools, see Ref. 1. 

Historically, the development of industrial organization has been the 
growth and transition of industry from one composed of countless small 
businesses into one characterized by large enterprises. Whereas the small 
companies were managed by single individuals, these individuals were no 
longer able to perform effectively all the managerial functions required for 
large enterprises. Consequently, there resulted a division of the manage- 
ment function so that responsibilities were shared by managers of produc- 
tion, sales and marketing, finance and control, personnel, and so forth. 
t With the further growth of industry, functional operations, such as pro- 
duction and marketing, were decentralized and resulted in a further division 
of the management function. For example, the production function was 
now carried on by a combination of subfunctions such as production plan- 
ning, material control, maintenance, purchasing, stock, and quality control. 
The production manager was elevated to an executive position somewhat 
removed from the actual production scene. 

This subdivision of the management function focused attention on prob- 
lems of various subfunctional areas of the industrial organization. Thus, a 
concentration on production problems by scientists and engineers led to the 
establishment of industrial engineering, chemical engineering, mechanical 
engineering, electrical engineering, and statistical quality control. Sciences 
such as industrial psychology and industrial sociology arose as a result of a 
concentration of interest on the solution of personnel problems, etc. 

However, with this increased concentration of attention to subfunctional 
problems (e.g., those of production or marketing), and the corresponding 
emergence of sciences devoted to the solution of these problems, a new type 
of problem arose — that concerned with the solution of industrial problems 
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in terms of the overall objectives of the organization rather than in terms 
of the objectives of each individual subfunctional unit. ' 

For example, in production scheduling and inventory control the produc- 
tion department wishes to minimize the cost of production and strives to 
achieve this by means of long production runs of a relatively few types of 
items. This leads to a large inventory (in order to achieve savings in setup ' 
costs) spread over just a few types of items. 

The sales department, on the other hand, is usually interested in sales 
volume and in being able to fill "every" customer order. They, too, would 
like to have a large inventory, but one which is composed of many types of 
items. Furthermore, they are usually in favor of a highly flexible produc- 
tion shop, one able to produce quickly to customer order rather thani one 
committed to long production runs. . ; 

The finance department is interested in the production scheduling and 
inventory control problem in terms of the capital investments and, conse- 
quently, is all too apt to seek low inventory levels. 

The personnel department is also concerned with the production schedul- 
ing and inventory control problem. It would like to use inventories as a 
means of stabilizing labor requirements, thereby reducing costs associated 
with hiring and layoff, fringe benefits, retraining, employee malcontent, etc. 

The Executive Problem. The objectives of each of these departments 
are not mutually consistent but, rather, are in conflict with one another. 
Therefore, the policy established by any one of these departments will sel- 
dom be favorable to the others. Consequently, there exists an executive 
problem — an overall problem of mediating among these conflicts of interest 
and selecting that course of action which is best for the organization as a 
whole. It is to the solution of this type of overall problem that the methods, 
techniques, and tools of O.R. are directed. The executive problem arises 
anywhere that there is a need for such a mediation of conflicts of interest 
in terms of the effectiveness of the overall system. Hence O.R. is not re- 
stricted to the solution of problems at the level of, say, the corporation 
president but rather at any functional or subfunctional level where such a 
conflict of interests exists. 

For many years there have been instances of the application of science 
to the solution of executive problems. However, the most significant stim- 
ulus was provided as a result of the military emergencies of World War II, 
especially in the Battle of Britain. Here, in a sense somewhat of despera- 
tion, scientists were called upon to solve "executive type" problems of 
strategy and tactics for the most effective utilization of very limited re- 
sources (Ref. 2). Interdisciplinary teams of scientists and engineers were 
formed to bring to bear the widest range of scientific knowledge and meth- 
odology to the solution of these problems. These teams, engaged in what 
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became known as Operational Research in Great Britain, were so successful 
that O.R. units were soon established for military services in the United 
States. 

Definition of Operations Research. One working definition that has 

been stated is: "Operations Research is the application of scientific methods, 

* techniques, and tools to problems involving the operations of a system so 

as to provide those in control of the system with optimum solutions to the 

problems" (Ref. 1, p. 18). 

Methodology of Operations Research. By the use of teams (members 
drawn from different disciplines), a variety of scientific methods, tech- 
niques, and tools is made available. O.R. personnel have developed a 
methodological approach to the solution of system problems consisting of 
the following steps: 

1. Formulating the problem. 

2. Constructing a mathematical model to represent the system under 
study. 

3. Deriving a solution from the model. 

4. Testing the model and the solution derived from it. 

5. Establishing controls over the solution. 

6. Implementing the solution. 

O.R. establishes a model to represent the system under study in order to 
be able to perform an abstract manipulation of the system (i.e., via the 
model) and determines how the system can best be changed.' 

Prototype Problems of Operations Research. In applying O.R. 
methods, techniques, and tools toward the solution of problems of the in- 
dustrial system, several classes of problems have arisen with great fre- 
quency. These recurrent problems gave rise to the development of special 
techniques for their solution. The principal classes of recurrent problems 
are : (a) inventory, (6) allocation, (c) waiting time or queuing, (d) replace- 
ment, (e) competition. 

For general articles on O.R. prior to 1954, see the bibliography of Ref. 2. 
See also Refs. 3 and 4. 

Phases of Operations Research 

Formulation of the Problem. This is usually a sequential process. 
An initial formulation is completed and research proceeds, but, in proceed- 
ing, the problem is subjected to almost continuous and progressive reformu- 
lation and refinement. This continues until a solution is reached. In a 
sense, one never knows until the end of the research whether the problem 
was correctly formulated, and perhaps not even then. 

Anxiety to get the research under way frequently leads to reduction of 
the time and effort devoted to formulating the problem. This is likely to 
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be very costly. Consequently, some systematic way of formulating the 
problem should be a standard procedure of an O.R. team, and a specific 
allocation of time for formulating the problem should be made. 

The procedure for formulating a problem may be summarized in the follow- 
ing outline: 

A. Analyze the relevant operations and the communication system by 
which they are controlled. 

1. Identify and trace each communication related to these operations. 

2. Identify each transformation of information and decision process. 

3. Identify each step in the operations. 

4. Drop from consideration each communication or transformation 
which has no effect on operations (e.g., billing in production opera- 
tions). 

5. Group operations between control points. 

6. Prepare a flow chart showing : (a) control points and decisions made, 
(6) flow of pertinent information between control points and time 
consumed, and (c) flow of materials and time of grouped operations. 

B. Formulate management's problem. 

1. Identify the decision-makers and the decision-making procedure. 

2. Determine the decision-makers' relevant objectives. 

3. Identify other participants and the channels of their influence on a 
solution. 

4. Determine the objectives of the other participants. 

5. Determine the alternative courses of action available to the decision- 
makers. 

6. Determine the counteractions available to other participants. 

C. Formulate the research problem. 

1. Edit and condense the relevant objectives. 

2. Edit and condense the relevant courses of action. 

3. Define the measure of effectiveness to be used: (a) define the measure 
of efficiency to be used relative to each objective, (b) weight objec- 
tives (if qualitative) or units of objectives (if quantitative), and (c) 
define the criterion of best decision as some function of the sum of 
weighted efficiencies (e.g., maximum expected return, minimum ex- 
pected loss). 

The Components of a Problem. These are: (1) the decision-maker, 
(2) objectives, (3) the system, and (4) alternative courses of action. 

The Decision-Maker. The following questions may serve to direct 
study of the decision-making process in the problem area : 

1. Who has the responsibility for making recommendations concerning 
modification of policies? 
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2. Whose approval is required and how is this approval expressed? 
3: What constitutes final approval? (A majority vote in group delibera- 
tion, approval by a final authority in*a sequence of reviews, etc.) 

4. Does anyone have absolute veto power? If not, how can a recommen- 
dation be rejected? 

5. Who has the responsibility for carrying out recommendations once 
they are approved? 

6. Who has responsibility for evaluating the action taken? 

' The Decision-Maker's Objectives. Direct questioning of decision- 
makers seldom reveals all their pertinent objectives relative to a problem. 
Such questioning provides a start, but it seldom provides enough informa- 
tion for a complete formulation of objectives. 

For example, objectives to be obtained may be: (a) to decrease produc- 
tion costs, (b) to render better customer service, (c) to increase a share of 
the market. Objectives to be retained may be: (a) to maintain stable em- 
ployment, (b) to retain product leadership, (c) to preserve good relations 
with the community. 

The System. Most organized systems involve the following compo- 
nents: (1) management, which directs (2) men, who control and operate 
(3) machines, which convert (4) materials into products or services made 
available to (5) consumers, whose purchases are also sought by (6) com- 
petitors, (7) government, and the public. 

Alternative Courses of Action. A number of possible alternative 
courses of action are ordinarily disclosed in the process of going through the 
earlier step's in formulating the problem. The list of alternatives disclosed 
in this way may not be, exhaustive. The researchers should get as com- 
plete a list of alternatives as possible, even to the extent of including possi- 
bilities that are not thought to be feasible. 

The derivation of a list of alternatives comes about by asking and answer- 
ing the following questions. For each phase of the system, would the effi- 
ciency be affected by a change in (1) personnel, (2) operations, (3) the 
materials and/or machines, or (4) environment? Whenever an affirma- 
tive answer is obtained, the specific alternatives at this stage can be ex- 
plored. 

The Research Problem. Transformation of the decision-maker's 
problem into a research problem involves the following steps: (1) editing 
the list of objectives obtained in the first stage of problem formulation, 
(2) editing the list of alternative courses of action, (3) defining the measure 
of effectiveness to be used, 

Before proceeding to a detailed discussion of each of these three steps, 
some remarks concerning the logic of decision-making (i.e., decision theory) 
are in order. 
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Example. Consider the following simplified abstract problem. Only- 
two objectives are involved, Oi and 2 ; and only two courses of action are 
possible, C\ and C 2 . Now suppose the efficiency of each course of action 
for each objective has been determined (on a scale from to 1) and the re- 
sults are shown in the following matrix: 





Oi 


o 2 


Ci 


0.8 


0.4 


c 2 


0.2 


0.6 



Which course of action should be selected? It is a mistake to answer 
either "Ci" or "C 2 ." The question cannot be answered without informa- 
tion concerning the relative importance of the objectives. If Oi is much 
more important than 2 , it seems clear that Ci should be selected, but if 2 
is much more important than O if C 2 should be selected. How can the cri- 
terion of selection be made explicit? If the relative importance of 0\ and 
2 could be measured, such a criterion could be provided. Suppose that 
relative importance could be measured along a scale running from to 1, 
and is 0.3 for 0\, and is 0.7 for 2 . Now the efficiency of each course of 
action for each objective can be weighted as follows: 





Oi 


o 2 


Total 


Ci 


0.3 X 0.8 = 0.24 


0.7 X 0.4 = 0.28 


0.52 


c 2 


0.3 X 0.2 = 0.06 


0.7 X 0.6 = 0.42 


0.48 



The sum of the weighted efficiencies (efficiency times relative importance) 
of a course of action is its relative effectiveness. 

Editing the Objectives and Courses of Action. The purpose of 
editing the objectives is to simplify and condense the list obtained in the 
first stage of formulating the problem. The editing procedure can be con- 
sidered in three steps. 

1. An examination of the list to determine whether the attainment of 
any one objective is important only because it is a means to the attainment 
of another objective or objectives. If so, it may be eliminated. For exam- 
ple, suppose one of the objectives is "to increase the company's annual net 
profit," and another is to "decrease production costs." It is likely that 
there is interest in decreased production costs only to the extent that it 
leads to increased net profits. If so, "decreased production costs" can be 
considered as a "means," not as an objective. 

2. Examination of each objective relative to the alternative courses of 
action to determine whether attainment of any of the objectives would be 
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unaffected by a choice from among the alternatives. If an objective is so 
unaffected, it should be eliminated from the list. For example, suppose one 
objective listed is "to maintain a high quality product" and the alternative 
courses of action involve only determination of production lot sizes. Then, 
if quality is unaffected by lot size, the "quality maintenance" objective 
can be dropped from the list. 

3. Combine objectives of different participants that are essentially the 
same. For example, both employer and employee may be interested in 
stable employment, and both manufacturer and consumer may be inter- 
ested in low price and high quality. 

Editing the Courses of Action. The list of possible alternative actions avail- 
able to the decision-maker should be examined to determine whether there 
are reasons for eliminating any of these from further consideration. 

Defining the Measure of Effectiveness. (See Ref. 1, Chap. 5.) 
Measures of effectiveness consist of two component measures: (1) the 
importance of the objectives and (2) the efficiency of the courses of 
action. 

The procedure for establishing an overall measure of effectiveness is a 
very complex one. However, for quantifiable objectives, this procedure may 
be summarized as follows: 

1. Develop a measure of efficiency relative to each objective. 

2. Where the measures of efficiency obtained in step 1 differ, develop a 
way of transforming the measures into one common or standard measure 
of efficiency. 

3. For each course of action and each objective, determine the probability 
of attaining each possible level of efficiency. This is the efficiency function 
of each course of action for each objective. 

4. For each course of action, "add" the efficiency functions so as to 
obtain a combined efficiency relative to all objectives. The result is an 
effectiveness function for each course of action relative to the entire set of 
objectives. 

5. State the objective of the decision process in terms of maximizing or 
minimizing expected return, gain, or loss. 

6. Construct a "return function" for each course of action. A return 
function expresses the expected outcome (outcome times its probability of 
occurrence) in terms of gains and losses. 

For establishing measures of effectiveness for mixed quantitative and 
qualitative objectives or for purely qualitative objectives, see Ref. 1, 
Chap. 5. 

Constructing a Model (Ref. 1, Chap. 7). A mathematical model 
expresses the effectiveness of the system under study as a function of a set 
of variables, at least one of which is subject to control. 
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The general form of an O.R. model can be expressed as 

E = ffa, Vj), 

where E represents the effectiveness of the system, Xi those variables of the 
system which are subject to control, and y } - those variables which are not 
subject to control. Restrictions, if any, on values of the variables may be 
expressed in a supplementary set of equations and/or inequations. The 
O.R. problem is to determine those values of the control variables, X{, which 
best achieve the given measure of effectiveness, E. 

The procedure for constructing a mathematical model may be summa- 
rized as follows: 

1. Itemize those components of the system which affect its effectiveness. 

2. Edit this list of components (combining and eliminating factors). 

3. Assign a symbol to each factor retained. 

4. Construct one (or more) equations which express the effectiveness of 
the system as a function of the pertinent variables. 

5. Identify the control variables, i.e., variables which can be manipula- 
ted by management decision. 

Deriving a Solution from the Model (see Sect. 2). There are es- 
sentially two types of procedures for deriving an optimum solution from a 
model, analytic and numerical. 

Analytic procedures consist of the use of mathematical deduction. 
Through the application of various branches of mathematics such as cal- 
culus and matrix algebra, analytic solutions are obtained in the abstract; 
i.e., the substitution of numbers for symbols is generally made after the 
solution has been obtained. 

Numerical procedures consist of trying various values of the control vari- 
ables in the model, comparing the results obtained, and selecting that set 
of values of the control variables which yield the best solution. Such pro- 
cedures vary from pure trial-and-error to iterative procedures in which 
successive intermediate trials converge to an optimum solution. 

In many instances, especially in probabilistic models, one may not be 
able to evaluate certain terms or expressions in the model because of mathe- 
matical or practical considerations. In many such cases, a particular appli- 
cation of random sampling, called the Monte Carlo technique, can be used to 
obtain accurate evaluations of the expressions. 

Testing the Model and the Solution (see Sect. 8). A model is never 
more than a partial representation of the system under study. It is a good 
model if, despite its incompleteness, it can accurately predict the effect of 
changes in the system on the overall effectiveness of the system. 

The solution can be evaluated by comparing results obtained without 
applying the solution with results obtained when it is used. These evalua- 
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tions may be performed retrospectively by the use of past data or by a trial 
run or pretest. Testing requires careful analysis as to the validity of the 
data. 

Establishing Controls over the Solution (see Sect. 9). A solution 
derived from a model remains a solution only as long as the uncontrolled 
variables retain their values and the relationship between the variables in 
the model remains constant. The solution itself goes "out of control" 
when the value of one or more of the uncontrolled variables and/or one or 
more of the relationships between variables have changed significantly. 
The significance of the change depends on (1) the amount by which the 
solution is made to deviate from the true optimum under the changed con- 
ditions and (2) the cost of changing the solution in operation. To establish 
controls over the solution, then, one must develop tools for determining 
when significant changes occur and rules must be established for modify- 
ing the solution so as to take these changes into account. 

Putting the Solution to Work (see Sect. 10). The tested solution 
must be translated into a set of operating procedures capable of being un- 
derstood and applied by the personnel who will be responsible for their use. 
Required changes in existing procedures and reserves must be specified and 
carried out. 

2. SOLUTION OF THE MODEL 

Types of Models. A scientific model is a representation of some subject 
of inquiry (such as objects, events, processes, systems) and is used for 
purposes of prediction and control. The primary function of a scientific 
model is explanatory rather than descriptive. It is intended to make pos- 
sible or to facilitate determination of how changes in one or more aspects 
of the modeled entity may affect other aspects, or the whole. In the em- 
ployment of models, this determination is made by manipulating the 
model rather than by imposing changes on the modeled entity itself. 

Models may be classified into three types (see Ref. 1, Chap. 7) : 

1. Iconic models pictorially or visually represent certain aspects of a 
system. Examples: a photograph, a scaled model plant layout. 

2. Analog models represent one set of (system) properties by another set 
of properties. Examples: graphs, analog computers. 

3. Symbolic models, by means of mathematical equations, employ sym- 
bols to designate properties of the system. Examples: the force of gravity, 
F = mg. 

Note. This section will deal only with symbolic models, since most prob- 
lems in O.R. are solved by means of such models. 

The model is an instrument which helps one evaluate alternative policies 
efficiently. The selection of a procedure for deriving a solution to the 
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problem from the model depends upon the characteristics of the model. 
These procedures can be classified into two types, analytic and numerical. 

Analytic procedures are essentially deductive in character, whereas numer- 
ical procedures are essentially inductive. In some cases (especially proba- 
bilistic situations), neither of these procedures can be applied until one or 
more terms in the equation have been evaluated by what is called the 
Monte Carlo technique. 

Analytic Solutions 

Symbolic models can assume a variety of mathematical forms and conse- 
quently require many different types of mathematical analyses for solution. 
The primary means of obtaining analytic solutions is through the use of 
calculus (see standard college textbooks). The methods of calculus are 
especially applicable to the solution of inventory problems (see Sect. 3). 

One Control Variable, No Restrictions. To illustrate the application 
of the methods of calculus, consider the elementary inventory situation 
described in Sect. 3 and represented by the model given there in eq. (1). 
See Sect. 3 for definition of symbols. 

The total expected relevant cost per year, TEC, is given by 

1 
(1) TEC = iC 1 Tq + -C s R, 

Q 

where q is the controllable variable which designates the number of items 
to be produced per run. The problem is to determine the most economic 
lot size, q*. Take the derivative of TEC, eq. (1), with respect to q: 



(2) 








d(TEC) 
dq 


ic,T- 


C S R 


Set eq. 


(2) 


equal 


to 


zero and solve for q: 




(3) 
Then 








ic x r- 

q* = 


C S R 


0. 


(4) 


IBC. 





Td 



represents the most economic number of items per run, i.e., that value of q 
which minimizes TEC. 

Two or More Control Variables, No Restrictions. When the model 
contains more than one control variable, one must take partial derivatives. 
Thus, if one wishes to determine the economic lot size for two independent 
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items by means of one equation, the total estimated relevant cost per year 
becomes 



(5) 



TEC = (iC n T qi + - C.a) + (\C X2 Tq 2 + - C s2 /A 



where q± and q 2 are the respective control variables. 

By taking the partial derivatives of TEC with respect to qi and q 2 , 



d(TEC) ' R 1 d(TEC) , fl 2 

(6) = iC u T - C sl — and -A- L = \C 12 T - C s2 -§ 

<% Qi dq 2 q<f 

which, when equated to zero, yield the most economic lot sizes: 



(7) ? '* = J2 ^T and ^ = V 2 1^ 

More generally, for any number of control variables, 



(8) ' RiCs 



TC U 



Two or More Control Variables with Restrictions. When one has 
an objective function which is to be maximized or minimized subject to a 
set of restrictions, a number of methods or techniques for solution may be 
applicable, but only one is discussed here. 

Lagrangian Multipliers. The method of Lagrangian multipliers en- 
ables one to solve the following problem. 

Problem. Determine values of x\, x 2 , • • •, x n which maximize 

(9) u = f(x lf x 2 , •••, x n ) 

subject to 



(10) 



gi(xi, x 2 , •••,&„) = o 

g 2 (x u x 2 , --,x n ) = 



Qm\X\, X 2 , ' ' ', X n ) — 

By the method of Lagrange, a new function is formed, 

(11) 4> = f(x lf X 2 , • • •, X n ) + \igifal, X 2 , •••, X n ) 

■+-•••+ h m g m \x\, x 2 , ' ' • , x n ), 

where Xi, X2, • • • , X m are undetermined multipliers called Lagrangian multi- 
pliers. 
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Then, in order to determine the extremal values of u = f(xi, x 2) ■ • •, x n ), 
all that is necessary is to obtain the solution of the system of eqs. (10) for 
the unknowns x i} x 2 , • • •, x n , Xi, X 2 , • • •, X OT . (See Ref. 5.) 

Example. Find the point in the plane x -f- 2y + 3z = 14 nearest the 
origin. The problem may be converted to that of finding values of x, y, 
and z which minimize the square of the sphere diameter 



subject to 
Form: 



u = D 2 = x 2 + y 2 + z 2 
g = x + 2y + 32 - 14 = 0. 



$ = x 2 -{- y 2 + z 2 - \(x + 2y + 3z- 14). 
Take partial derivatives of 4> with respect to x, y, z, and X. 

— = 2x - X, — = 2y - 2X, 

dx dy 

dd> d(j> 

— = 2z - 3\, — = a; + 2?/ + 32 - 14. 

as ax 

Setting these four partial derivatives equal to zero and solving the system 
of four simultaneous equations then yields: 

x = 1, y — 2, 2 = 3, and X = — 2; 

that is, the point in the plane x -\- 2y -\- 3z = 14 nearest the origin is 
(1, 2, 3). 

Other examples of the use of Lagrangian multipliers can be found in 
Ref. 6 and in texts on advanced calculus. 

Modified Lagrangian Multiplier Method (Ref. 1, Chap. 10). Many 
practical extremal problems have the added restriction that all variables 
must be non-negative; for example, it makes no sense to produce — n units 
of a given product. Furthermore, the restrictions may be given in the form 
of inequalities instead of equalities. Since the Lagrangian multiplier 
method does not guarantee the non-negativity of the solution variables, a 
modification must be made. 

Example. Consider an economic lot size inventory problem (of the 
type described above) involving two products, with a restriction on the 
total available warehouse space. If W\ and W 2 are the respective unit 
storage requirements, and an average inventory level is assumed equal to 
one-half the lot size q, the total space requirement can be written as 

(12) iSTFtf,- = iW l9l + iW 2 q 2 = S. 
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If W x = 5 cu ft, W 2 = 35 cu ft, and S = 14,000 cu ft, eq. (12) becomes 

(13) hi + ¥92 ^ 14,000, 
or, equivalently, 

(14) 5gi + 35g 2 ^ 28,000. 

The problem (for two products) can then be stated as: 

Problem. Determine non-negative values of q\ and q 2 which minimize 

TEC = (iC u Tqi + - CiR^ + (%C 12 Tq 2 + - C. 2 R^j 

subject to the restriction of eq. (14). 
Solution. Define an undetermined multiplier X such that 

X < when S - i2W iqi = 0; 

(15) 

X = when S - |2T^g; > 0. 

Form 

(16) 4>=TEC + X(S - i2W<q<), 
that is, 

(17) = (iC n Tqi + - «) + ( \C X2 Tq 2 + - C s2 R 2 ) 

\ gi / \ q 2 / 

+ X(S - jTTrfi - jF 2 g 2 ). 

Since X(£ — MtWili) is always identically zero by eq. (15), <£ = TIEC. 
Taking partial derivatives of <f> with respect to gi and q 2 yields 

d(TEC) , 1 

(18a) = iC n T - — 2 C sl R t - %\W i, 

331 gi 

and 

a(r^C) , 1 

(18b) - = \C X2 T - — C s2 R 2 - i\W 2 . 

dq 2 <l2 

Setting eqs. (18) equal to zero yields 



(19a) 
and 

(196) 





/ 2C S \R\ 


<Zi — , 


\C n T - \Wt 




1 2C s2 R 2 


°2 


\C l2 T - \W? 
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For each product, the quantities Ri, C a i, Cu, W{, and T are known, but 
X is still unknown. However, for any arbitrarily assigned value of X, q { 
and, hence, y^Wiqi can be calculated. If Y^Wiqi exceeds *S (see eqs. 12 
and 13), the lot sizes are too large. In this case, decrease X repeatedly and 
recompute until Y^Wiqi = S has been obtained. If Y^Wiqi < S for all 
negative X, set X = in eq. (19). The resulting g/s will allow the smallest 
possible total costs for the company with existing warehouse space S. 

Table 1. Storage Set by Various X Values 

X qi* q 2 * £(5gi + 35 ?2 ) 

-0.0000 ' 816 756 15,270 

-0.0012 813 721 14,650 

-0.0024 810 690 14,100 

-0.0036 806 663 13,618 

-0.0060 800 617 12,790 

-0.0084 794 580 12,135 

-0.0120 784 535 11,323 

a Assumes: T = 12 months and 

Product Ri Cai Cu 

Xi 2400 $100 0.060 

X 2 4800 $ 25 0.035 

Values of 3^(5gi + 35g 2 ) are calculated in Table 1 in order to determine 
the correct value of X. As indicated in Table 1, X should be approximately 
equal to —0.0024 so that 

qi* = 810 and q 2 * = 690. 

Note. For this example, without any restriction on storage space (mini- 
mizing TEC, rather than #), 

gi* = 816 and q 2 * = 756. 

Another approach to a modified Lagrangian multiplier technique can be 
found in Ref. 7. Such modified Lagrangian multiplier techniques, when 
applicable, are most cumbersome and impractical for a large number of 
variables. (See Ref. 1, Chap. 10.) Where the objective function and the 
restrictions are linear (and for some other special cases), the techniques of 
linear programming are applicable (see Sect. 4). 

Other Analytic Methods of Solution. There are many other analytic 
methods of solution much more sophisticated than those presented here. 
A number of the models arising in specific problems require the develop- 
ment of special methods for their solution. For these more sophisticated 
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and special methods, see journals such as Operations Research, Management 
Science, Econometrica, Naval Research Logistics Quarterly, and the publica- 
tions of the RAND Corporation (Santa Monica, Calif.). See also Refs. 7, 
8, and 9. 

Numerical Solutions 

Numerical techniques of deriving a solution from a model consist of 
substituting numbers for the symbols in the model and finding that set 
of substituted numbers which yields the maximum effectiveness. Some 
numerical procedures are trial-and-error procedures into which one seeks 
to build some rationale for the selection of subsequent trials. Others are 
so-called iterative procedures in which one converges to an optimum solu- 
tion through successively better steps. 

Newton's Method. An example of a quite useful trial-and-error 
procedure is Newton's method for solving equations, which is a procedure 
for determining, within any desired degree of accuracy, the roots of an 
algebraic equation. The method is based on the fact that, for a short dis- 
tance, the tangent to a smooth curve is a good approximation to the curve. 

Newton's method may be formulated as follows. Let /(X) = be the 
equation under consideration. A root of this equation is the abscissa of a 
point at which the curve Y = /(X) crosses the X-axis. 

Start with a trial solution, say X (see Fig. 1). This value X determines 
a point P on the curve whose coordinates are (X , F ). The tangent to the 



Y=f(X) 




Fig. 1. Figure for Newton's method (Ref. 1). 

curve at P is then drawn and will intersect the X-axis at (Xi, 0). If the 
curve and the tangent are nearly coincident over the range (X , Xi), the 
value Xi will be the first approximate root of the equation. Furthermore, 
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using the fact that the slope of the tangent at P is given by/'(Xo), namely 
the derivative of f(X) evaluated at X = X , yields 

(20) Xi = X - ^~^- 

/'(X ) 

The procedure may be repeated as many times as necessary where, in 
general, 

f(Xn) 

(21) X n A-i = X n • 

Whether and how fast the process will converge depends on the function 
f(X) and the initial value X . Conditions favorable to convergence are 
evidently that/(X ) be small and/'(X ) be large. 
To illustrate Newton's method, consider 

f(X) = X 3 - 3X 2 + 4X - 2. 

Although there are many devices which can be used to locate integers be- 
tween or at which roots will lie, arbitrarily take X = 2 as the trial solution. 
For the particular f(X) chosen, X = 1 is obviously a solution, and that is 
what we wish to approximate by Newton's method. The deviation from 
the value X = 1 will, of course, measure the degree of accuracy of this 
approximation. 
Now 

f(X) = 3X 2 - 6X + 4, 
so that 

/(2) =8- 12 + 8-2 = 2, 

/'(2) = 12 - 12 + 4 = 4. 

Hence, using eq. (21) yields 

Zi = 2 - f = 1.5. 

By continuing in this manner, 

/(1.5) = (f ) 3 - 3(f) 2 + 4(f) -2 = f, 



and 

f (1.5) = 3(f) 2 - 6(f) +4 = 1, 
so that 

X 2 = 1.5 - -& = 1.143. 
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Continuing once more gives 

/(1.143) = 0.147, 



so that 



/'(1.143) = 1.060, 



0.147 

X 3 = 1.143 = 1.004. 

1.060 



One could continue in this manner, measuring at each stage of the itera- 
tive procedure the value of f(X{) to indicate how quickly one is converging 
to a solution [obviously, at a point of solution X*, f(X*) = 0], and, hence, 
obtain this solution within any prescribed degree of accuracy. 

Excellent examples of converging iterative procedures are to be found in 
the several techniques of linear programming. These are discussed in 
Sect. 4. 

The Monte Carlo Technique 

In many mathematical models, it is necessary to evaluate certain terms 
in the model before a solution can be derived. Especially where probabil- 
ity concepts are involved, it may not be possible or practical to evaluate a 
given function (within a model) by mathematical analysis. Such expres- 
sions, however, can be evaluated by the Monte Carlo technique. Specifi- 
cally, the Monte Carlo technique is a procedure by which one can obtain 
approximate evaluations of mathematical expressions which are built up 
of one or more probability distribution functions. 

The Monte Carlo technique consists of simulating an experiment to 
determine some probabilistic property of a population of objects or events 
by the use of random sampling applied to the components of the objects 
or events. This statement can best be clarified by means of examples. 

The Random Walk Problem. The discovery of the Monte Carlo 
technique is said to be due to a legendary mathematician observing the 
perambulation of a saturated drunk. The mathematician wondered how 
many steps the drunkard would have to take, on the average, to reach a 
specified distance from his starting point, if it were assumed that, at each 
step, there was an equal probability of the drunkard stepping off in any 
direction. 

Example. To illustrate how the Monte Carlo technique can be applied 
to this problem of the "random walk," an estimate can be obtained of the 
probable distance traveled after five steps of equal size. (It is further as- 
sumed, for simplicity of presentation, that these steps are at 45°, 135°, 
225°, or 315°.) To do this, refer to Table 2, which is a portion of a two- 
digit random number table. 
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Table 2. Random Numbers (Ref. 1) 

09 73 25 33 76 53 01 35 86 34 67 35 48 76 80 95 90 90 17 39 29 27 49 

54 20 48 05 64 89 47 42 96 24 80 52 40 37 20 63 61 04 02 00 82 29 16 
42 26 89 53 19 64 50 93 03 23 20 90 25 60 15 95 33 47 64 35 08 03 36 
01 90 25 29 09 37 67 07 15 38 31 13 11 65 88 67 67 43 97 04 43 62 76 
80 79 99 70 80 15 73 61 47 64 03 23 66 53 98 95 11 68 77 12 17 17 68 

06 57 47 17 34 07 27 68 50 36 69 73 61 70 65 81 33 98 85 11 19 92 91 

06 01 08 05 45 57 18 24 06 35 30 34 26 14 86 79 90 74 39 23 40 30 97 

26 97 76 02 02 05 16 56 92 68 66 57 48 18 73 05 38 52 47 18 62 38 85 

57 33 21 35 05 32 54 70 48 90 55 35 75 48 28 46 82 87 09 82 49 12 56 

79 64 57 53 03 52 96 47 78 35 80 83 42 82 60 93 52 03 44 35 27 38 84 

52 01 77 67 14 90 56 86 07 22 10 94 05 58 60 97 09 34 33 50 50 07 39 

80 50 54 31 39 80 82 77 32 50 72 56 82 48 29 40 52 42 01 52 77 56 78 
45 29 96 34 06 28 89 80 83 13 74 67 00 78 18 47 54 06 10 68 71 17 78 
68 34 02 00 86 50 75 84 01 36 76 66 79 51 90 36 47 64 93 29 60 91 01 

59 46 73 48 87 51 76 49 69 91 82 60 89 28 93 78 56 13 68 23 47 83 41 

48 11 76 74 17 46 85 09 50 58 04 77 69 74 73 03 95 71 86 40 21 81 65 
12 43 56 35 17 72 70 80 15 45 31 82 23 74 21 11 57 82 53 14 38 55 37 
35 09 98 17 77 40 27 72 14 43 23 60 02 10 45 52 16 42 37 96 28 60 26 
91 62 68 03 66 25 22 91 48 36 93 68 72 03 76 62 11 39 90 94 40 05 64 
89 32 05 05 14 22 56 85 14 46 42 75 67 88 96 29 77 88 22 54 38 21 45 

49 91 45 23 68 47 92 76 86 46 16 28 35 54 94 75 08 99 23 37 08 92 00 
33 69 45 98 26 94 03 68 58 70 29 73 41 35 53 14 03 33 40 42 05 08 23 

10 48 19 49 85 15 74 79 54 32 97 92 65 75 57 60 04 08 81 22 22 20 64 

55 07 37 42 11 10 00 20 40 12 86 07 46 97 96 64 48 94 39 28 70 72 58 

60 64 93 29 16 50 53 44 84 40 21 95 25 63 43 65 17 70 82 07 20 73 17 

19 69 04 46 26 45 74 77 74 51 92 43 37 29 65 39 45 95 93 42 58 26 05 

47 44 52 66 95 27 07 99 53 59 36 78 38 48 82 39 61 01 18 33 21 15 94 

55 72 85 73 67 89 75 43 87 54 62 24 44 31 91 19 04 25 92 92 92 74 59 

48 11 62 13 97 34 40 87 21 16 86 84 87 67 02 07 11 20 59 25 70 14 66 
52 37 83 17 73 20 88 98 37 68 93 59 14 16 26 25 22 96 63 05 52 28 25 

49 35 24 94 75 24 63 38 24 45 86 25 10 25 61 96 27 93 35 65 33 71 24 
54 99 76 54 64 05 18 81 59 96 11 96 38 96 54 69 28 23 91 23 28 72 95 
96 31 53 07 26 89 80 93 54 33 35 13 54 62 77 97 45 00 24 90 10 33 93 
80 80 83 91 45 42 72 68 42 83 60 94 97 00 13 02 12 48 92 78 56 52 01 
05 88 52 36 01 39 09 22 86 77 28 14 40 77 93 91 08 36 47 70 61 74 29 

17 90 02 97 87 37 92 52 41 05 56 70 70 07 86 74 31 71 57 85 39 41 18 
23 46 14 06 20 11 74 52 04 15 95 66 00 00 18 74 39 24 23 97 11 89 63 

56 54 14 30 01 75 87 53 79 40 41 92 15 85 66 67 43 68 06 84 96 28 52 
15 51 49 38 19 47 60 72 46 43 66 79 45 43 59 04 79 00 33 20 82 66 85 
86 43 19 94 36 16 81 08 51 34 88 88 15 53 01 54 03 54 56 05 01 45 11 

08.62 48 26 45 24 02 84 04 44 99 90 88 96 39 09 47 34 07 35 44 13 18 

18 51 62 32 41 94 15 09 49 89 43 54 85 81 88 69 54 19 94 37 54 87 30 
95 10 04 06 96 38 27 07 74 20 15 12 33 87 25 01 62 52 98 94 62 46 11 
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Use the following symbolism : 

1. The lamppost is represented by the origin of the X- and F-axis. 
Fig. 2. 



See 




h ("3' 1) 



Fig. 2. Plotting of points (x n , x y ) (Ref. 1). 

2. The first digit of the two-digit random number selected from the table 
represents one unit of X, positive if even or zero, negative if odd. 

3. The second digit of the same two-digit random number selected repre- 
sents one unit of Y, positive if even or zero, and negative if odd. 

4. (x n , y n ) represents the position of the drunkard at the end of the nth 
phase. 

5. d n equals the distance of the drunkard from the lamppost at the end 
of the nth phase ; that is, dn = x n 2 + Vn 2 - 

To start at random, select the two-digit number, say in column 10 and 
row 6 of Table 2, and, by reading down, obtain the following five numbers: 
36, 35, 68, 90, and 35. These numbers may then be arranged and the 
drunkard's moves obtained as shown in Table 3. The points (x n , y n ) may 
also be plotted as in Fig. 2. 

Table 3 









Point 


Phase 


First 


Second 


Location 


n 


Digit 


Digit 


(Xn } Vn) 


1 


3 


6 


(-1, 1) 


2 


3 


5 


(-2, 0) 


3 


' 6 


8 


(-1, 1) 


4 


9 





(-2, 2) 


5 


3 


5 


(-3, 1) 



OPERATIONS RESEARCH 15-21 

In this example, then, one estimate is that the drunkard will be 3.16 units 
from the lamppost at the end of the fifth phase. This is obtained as follows : 

d 5 2 = X5 2 + Vs 2 , 
d 5 2 = 9 + 1, 
d 5 = VlO = 3.16. 

This procedure must be repeated for different random numbers in the 
table so that a group of estimates of the desired distance is obtained. The 
estimates in this group can then be averaged to yield an average estimated 
distance from the lamppost. In general, the estimates will improve as the 
number of such samples is increased. The accuracy of the estimate will be 
proportional to the square root of the number of samples. 

More generally, from many such simulated trials, the probability of the 
drunkard's being a specified distance from the lamppost for any number n 
of irregular zigzag phases is estimated. 

As a point of interest and as a basis for the reader comparing his own 
Monte Carlo solutions, it might be pointed out that, for this example, an 
analytic solution is obtainable and is given by 

d n = a-y/n, 

i.e., the most probable distance of the drunkard from the lamppost, after a 
large number of irregular phases of his walk, is equal to the average length a 
of each straight track he walks, times the square root of the number n of 
phases of his walk. 

For an illustration of the use of the Monte Carlo technique for the solu- 
tion of problems involving normal distributions, see Sect. 6. 

The use of the Monte Carlo technique for any probability distribution 
function can be found in Ref. 1, Chap. 7. Reference 1 discusses only the 
normal distribution, but the treatment is general and applicable to any 
probability distribution function. For a discussion of the nature of tables 
of random numbers and a bibliography of tables and works on this subject, 
see Ref. 10. Examples of other uses of the Monte Carlo technique can be 
found in Ref. 1, Chaps. 7, 14, and 17. See also Refs. 5, 11-13. 

3. INVENTORY MODELS 

Problem Statement. Inventory problems are concerned with mini- 
mizing the sum of costs such as those due to (a) carrying inventory, (6) 
setup, (c) shortage, (d) obsolescence, and (e) change of work force level. 
Inventory problems require the determination of (a) how many (or much) 
to order (i.e., produce or purchase) and/or (6) Avhen to order. 
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This section will introduce the kind of analysis that yields symbolic 
models of inventory processes. The mathematical models and solutions 
presented here pertain to specific inventory situations and progress from 
the most elementary to somewhat complex ones. For a complete definition 
and classification of the characteristics of inventory problems, see Ref. 1, 
Chap. 7. 

Decisions. The general class of inventory problems to be considered 
involves decisions concerning inventory levels. These decisions can be 
classified as follows: (1) The time at which orders for goods are to be placed 
is fixed. The quantity to be ordered must be determined. (2) Both the 
order quantity and order time must be determined. 

Cost. The costs associated with inventory are of three types: (1) setup 
cost, the fixed cost per lot of obtaining goods (purchasing or manufactur- 
ing) ; (2) inventory holding cost, including cost of money spent in obtaining 
the part, storage, obsolescence, handling, taxes, and insurance; (3) shortage 
cost, cost resulting from a delay in supplying the goods or an inability to 
fill the order at the time of request. , . 

Variables. The three major classes of variables in an inventory problem 
are: (1) cost variables, (2) demand variables, i.e., relative to customer de- 
mand for goods; (3) order variables, i.e., relative to obtaining the necessary 
goods. 

Elementary Inventory Models (see Ref. 1, Chap. 8) 

Symbols. The following symbols are used throughout the discussion of 
the elementary inventory models. 

q input, or quantity ordered 

qi input which occurs at the beginning of the ith time interval 

q* optimum order quantity 

r requirements per time interval 

n requirements for the ith time interval 

S inventory level 

Si inventory level at beginning of ith interval 

S{ inventory level at end of ith interval. Note. Si = Si — r,, and Si = 

Si -i + qi 

S* optimum inventory level at the beginning of a time interval 

t an interval of time 

t s interval between placing orders, in units of time 

t s * optimum interval between placing orders 

T period for which a policy is being established 

R total requirement for period T 

C\ holding cost per unit of goods for a unit of time 

Ci shortage cost per unit of goods for a specified period 

C s setup cost per production run 

TEC total expected relevant cost 
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TEC* 

P(r) 

f(r) 

P{r S S) 

F(r) 

F(S) 



minimum (optimum) total expected relevant cost 
probability of requiring r units, where r is a discrete variable 
probability density function of r, where r is a continuous variable 
probability of requiring S units or less, where r is a discrete variable 
cumulative probability function of r, where r is a continuous variable 



J 

Jo 



f(r) dr, probability of requiring S or less units, where r is a continu- 
ous variable. 

Model I. (See Fig. 3.) Given: (a) Demand is fixed and known, (b) 
Withdrawals from stock are continuous and at a constant rate, (c) No 
shortages are permitted. 

The variable costs are: Ci and C s (see Symbols above). 





-H 



Fig. 3. Inventory situation for Model I (Ref. 1). 

Problem. To determine: (1) how often to make a production run; (2) 
how many units should be made per run. 
Cost Equation. 

1 
TEC = iC 1 Tq + -C s R. . 
Q 



(22) 
Solution. 



(23) 



(24) 



q* = 



RC S 
TC~i 

~TC S 
RC X 



TEC* = V2RTdC s 



(25). 

Note that Model I is a special case of Model II, wherein C 2 = <x>. Ac- 
cordingly, by letting C 2 — > «> in eqs. (27-30), one readily obtains eqs. (23- 
25)- 
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Model II. (See Fig. 4.) Given: (a) Demand is known and fixed. (6) 
Shortages are permitted. 





-H 



Fig. 4. Inventory situation for Model II (Ref. 1). 

Cost Equation. 

1 9 (q-S) 2 C 2 T C S R 



(26) 
Solution. 

(27) 
(28) 
(29) 
(30) 



2q 


j w-i -r 


2q 


<f = y 


J TC 1 * 


lCx + c 2 

V ^2 


s*= ^ 


1 RC S 
\l TC X A 


/ C » , 


S/Ci + c a . 


• , 


l* TC \ 

\j RCt * 


/Ci + c 2 




/ To 


r<* — a 


/qpt^j 


^ / °2 



+ c 2 



Model III. Given: (a) Estimated variable demands and inputs, (b) dis- 
crete units, (c) shortages permitted (finite cost of shortage), (d) discontinu- 
ous distribution over time of withdrawals and input at a discontinuous 
rate, (e) known and constant reorder cycle time. 

In this model and in Model VI, the cost of carrying an inventory of parts 
until they are used is not taken into consideration. Rather, in this elemen- 
tary inventory situation, the cost of having excess parts that are never used 
is balanced against the cost of being short of parts when needed. 

Problem. To determine how many units of a given part should be ordered 
at the time of the initial purchase order. Here, one is balancing the cost of 
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having excess parts that are never used against the cost of being short of 
parts when needed. No consideration is given to the cost of carrying the 
inventory of parts until they are used. 
Cost Equation. 

S 00 

(31) TEC(S) = Ci £ P(r)(S - r) + C 2 £ P(r)(r - S). 

r=0 r=S+l 

Solution. The optimum value, S*, is given by that value of S which 
satisfies the inequalities: 

(32) P(r£S-l)< n ' < P(r £ S). 

w -+- C2 

For further discussion, derivation of this solution, and an example of its 
use, see Ref. 1, Chap. 8. 

Model IV. Given: (a) Estimated variable demand and inputs; (b) con- 
tinuous (rather than discrete) units; (c) shortages permitted, i.e., finite 
cost of shortage; (d) continuous distribution over time of withdrawals and 
input at a continuous rate; (e) known and constant reorder cycle time; (/) 
negative orders, i.e., returns, not considered. 

Problem. To determine the initial order quantity, where one balances 
the holding cost against the shortage cost. 

Cost Equation. 

(33) TEC(S) = C l f (S- r)f(r) dr + C 2 f (r - S)f(r) dr. 

Jo Js 

Solution. (See Ref. 1, Chap. 8.) The total expected cost is minimum for 
that value $ which satisfies the condition 

r s Co 

(34) F(S) S /(r) dr = - 

Jq L\ 



+ c 2 

Model V. Given: Conditions of Model IV plus a significant reorder 
lead time, i.e., one must take into account the lapse of time between the 
placing of an order and the receipt of the goods. 

Problem. To determine how much (many) should be ordered for the /cth 
day hence (where the reorder lead time is k days). 

Cost Equation. 

Let k = number of days in the order lead time, 

s = the stock level at the end of the period preceding the 
placing of the order, 
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Qii Q2, " ', Qk— i = quantities already ordered and due to arrive on the 
1st, 2nd, • • • , (k — l)st days hence, 
qk = quantity to be ordered for delivery k days hence, 

k 

R' = E r ii the total requirement over the order lead time, 

t=i 
S' = total of amount available in stock at end of previous 

period and amounts ordered over the present 7-day 

period; i.e., 

A— 1 

S' = s -f £ Qi + qk. 
*=i 

The problem is to determine the value of qk which will minimize the total 
expected cost over the lead time period, i.e., k days. However, since orders 
in the amounts qi, q 2 , • • • , g&-i have already been placed, the total expected 
cost for the first k — 1 days has already been determined and is no longer 
subject to control. Hence, equivalently, the problem is one of determining 
the value of qk which will minimize the total expected cost for the kth. day 
only. 

Solution. The stock at the end of the fcth period can be expressed as 

k—l k 

(35) s k = s + 2 Qi + Qk ~ 2 r i: 

z'=l i=l 

Then, since 

fc-i 

(36) S' = s + E Qi + Qk, 

;=i 
and 

k 

(37) R' = £ n , 

i=l 

the total expected cost for the Mh day will be given by 

OS' - R')f(R') dR f + C 2 I (R f - S')f(R') dR'. 
o Js' 

Equation (38) is equivalent to eq. (33) ; therefore the optimum value of 
S' is given by (see eq. 34) 

r s ' c 2 

(39) F(S') = f{R') dR' = — - 

Once having determined the optimum value of S r , namely S'*, qk* can be 
determined from eq. (36), i.e., 

(40) q k * = S'* ~ (so + E ff<)- 
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See Ref. 1, Chap. 8, for further discussion and an example of the use of 
this solution. 

Model VI. (See Fig. 5.) Given: Conditions of Model III except that 
withdrawals from stock are continuous and at a constant rate. 




r 



S-r 




(a) 

Fig. 5. Illustration for Model VI (Ref. 1). 

Problem. To determine hoAV many parts should be ordered at the time of 
the initial purchase order. 

Cost Equation, (a) For r ^ S. 

For a given value of r, the average number of units in stock over the 
order cycle period is given by 



(41) 



§[S+(S-r)]=*S--- 



The expected cost, for a specific value of r, (r ^ S), will then be 
(42) CiP(r)[S - 4r]. 

Therefore, the total expected cost for all r ^ S will be 



(43) 



Ci H P(r)(S - ir). 

r=0 



Cost Equation, (b) For r > S. 

Here, as seen from Fig. 5, there will be no shortages ti/(ti + t 2 ) part of 
the time, while shortages will occur t 2 /{h + t 2 ) part of the time. Now 



(44) 



h S t 2 

= - and 

h + t 2 r ti + k 



S 



Furthermore, the average amount stocked is %S, and the average amount 
short is Y^ir — S). Therefore, the holding cost for each value of r over 
the period S/r is given by 



(45) 



-©©->■■ 
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while the shortage cost for each r, over the period (r — S)/r, is given by 
. w - . (r-S) 2 

(46) „ . w . „ 



*rac-^-» 



2r 



Therefore, the total expected cost will be given by 

S oo o2 

(47) T#C(,S) = d £ P(r) OS - }r) +d E P(r) - 

r=0 r=S+l 2r 

^ (r-S) 2 

+ C 2 Z P(r) • 

Solution. The optimum value of $ is that which satisfies the condition 

(48) .. P[r £ (S - 1)] + (S - i) £ — < tt^TT 

< P(r 5; £) + (£-}- §) £ -^ 

See Ref. 1, Chap. 8, for further discussion, an example of the use of this 
solution, and a case study employing this model. 

Inventory Models with Price Breaks 

In this section, decision rules are given for the optimum lot size (or opti- 
mum purchase quantity) as derived for a class of inventory problems in 
which the unit manufacturing (or purchase) cost is variable, that is, sub- 
ject to quantity discounts or price breaks. Specifically, this section will 
generalize on Model I (see Elementary Inventory Models), which describes 
a system in which demand is fixed and known, withdrawals from stock are 
continuous and at a constant rate, and no shortages are permitted. (See 
Fig. 3.) 

Symbols. The following symbols are used: 

k{ cost per unit of manufacturing or purchasing for range i 

P monthly holding cost expressed as a decimal fraction of the value of the unit 

C s setup cost per production run or, when for purchased parts, the setup cost 

associated with the procurement of the purchased items 
TEK total expected cost 

TEK* minimum (optimum) total expected cost 

As before, 

T the period of time for which the decision rules are being determined 

R total requirement during period T 

t s interval between placing orders 

q input, or quantity ordered 

q* optimum order quantity, i.e., economic lot size or economic purchase quantity 
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Finally, let the price break situation be described by the following: 



Range 


Quantity 


Unit P 


'urch 


Ri 

i?2 


1 ^ qi < h 
h ^ Q2 < b 2 







tin On— I ^ Qn Kn 

where bj (j = 1, 2, • • •, n — 1) are those quantities which determine the 
price breaks. 

Problem. The problem can be stated as one of determining: (1) how 
often should parts be purchased; (2) how many units should be purchased 
at any one time. 

Basic Cost Equation. The basic cost equation for the period T for any 
one value of the unit purchase cost k x is given by 

(49) TEK = — + hR+ iCsTP + ihTPq, 

while the basic solution is given by 



j2C s R 
(50) q* = ^ , 

*'k x TP 

and 



(51) TEK* = V2hTPC s R + kjt + JC S TP. 

Solution. Decision Rules. (See Ref. 1, Chap. 9.) 

One Price Break. 

1. Compute q 2 * from eq. (50), by using k 2 . If q 2 * ^ b, then the opti- 
mum purchase quantity is q 2 *, that is, q* = q 2 *. 

2. If q 2 * < b, compute TEK*{k x ) from eq. (51) [or, equivalently, 
TEK(q x *) from eq. (49)] and compare this with TEK(b x ) as given by 
eq. (49). 

If TEK(q x *) < TEK(b x ), then q* = q x *. 
If TEK(q x *) > TEK(b x ), then q* = b x . 

Two Price Breaks. 

1. Compute q 3 *. If q 3t0 ^ b 2 , then q* = q 3 *. 

2. If q 3 * < b 2 , compute q 2 *. If q 3 * < b 2 and b x ^ q 2 * < b 2 , proceed as 
in the case of one price break, i.e., compare TEK*(k 2 ) with TEK{b 2 ) to 
determine the optimum purchase quantity. 

3. If (fe* < b 2 and q 2 * < b x , compute TEK*(k x ) and compare it with 
TEK(b x ) and TEK(b 2 ) to determine the optimum purchase quantity. 
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(n — 1) Price Breaks. 

1. Compute q n *. If q n * ^ & n -i, then q* = q n *. 

2. If q n * < 6„_i, compute g n _i*. If g n _i* ^ 6 n _ 2 , i.e., 6 n _ 2 ^ g„_! 
< 6 n _i, proceed as for one price break, i.e., compare TEK*(k n _i) with 
TEK{b n _i) to determine g*. 

3. If q n -i* <b n _2, compute q n -2*- If Qn-2* ^ &n-3, proceed as for 
two price breaks, i.e., compare TEK*{k n _ 2 ) with TEK(b n _ 2 ) and 
TEK(b n _i) to determine g*. 

4. If g n _ 2 * < &n-3, compute q n -s*- If #n-3* ^ K-4, compare 
TEK*(k n _ 3 ) with TEK(b n _ 3 ), TEK(b n _ 2 ), and TEK(b n _ x ). 

5. Continue in this manner until g n -i* ^ 6 n — y— i, (0 ^ i ^ n — 1), 
and then compare TEK*{k n _ 3 ) with TEK(b n _ 3 ), TEK(b n _ j+1 ), •••, 
TEK(b n _i) to determine the economic purchase quantity g*. A^o/e. De- 
fine & = 1 for this step. 

Inventory Models with Restrictions 

In some inventory situations it is necessary to consider restrictions on 
production facilities, storage space, time, or money. When such restric- 
tions are introduced in situations involving more than one product, it is 
necessary to allocate the limited available resources among the products. 
Models have been developed which enable one to determine how much of 
each item to produce (or purchase) under the specified restrictions. Such 
models are developed and solved in Ref. 1, Chap. 10. A brief description 
of the approach to the solution of such models is given in Sect. 2, Modified 
Lagrangian Multipliers. See also Refs. 14-16. 

Other Inventory Models 

Arrow, Harris, and Marschak (Ref. 17), Eisenhart (Ref. 18), Tompkins 
(Ref. 19), and others have treated the problem of determining the optimum 
buffer stock needed to protect against shortages, where demand is uncer- 
tain. Whitin (Ref. 20) has investigated the interaction between buffer 
stocks and lot sizes. Dvoretzky, Kiefer, and Wolfowitz (Refs. 21 and 22) 
have shown the conditions under which optimum inventory levels can be 
found. 

Multistorage Points. Berman and Clark (Ref. 23) have developed 
and solved specific models for systems in which a central warehouse sup- 
plies a number of field warehouses which, in turn, supply distributors. 

Dynamic Models. The dynamic problem of inventory is one in which 
consideration must be given to the effect of a decision in the current period 
on subsequent periods. 

A servomechanism approach to the dynamic inventory problem which 
utilizes feedback rules to adjust production to sales has been developed 
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and applied at Carnegie Institute of Technology (Refs. 8 and 24) for situa- 
tions of uncertain demand. This procedure applies Norbert Wiener's auto- 
correlation methods (see Chap. 17). A related method has been developed 
by Vassian (Ref. 26). 

A number of persons have developed approaches with linear program- 
ming techniques. Such linear programming models are designed primarily 
for situations with important seasonal fluctuations in demands. Charnes, 
Cooper, and Farr (Ref. 27) have treated this case while further assuming 
that demand is known. See also Dannerstedt (Ref. 28). 

Bellman (Refs. 29-32) has developed "dynamic programming" which 
makes it possible to approach these problems through the calculus of vari- 
ations. See also Bellman, Glicksberg, and Gross (Ref. 25). Holt, Modi- 
gliani, and Simon (Ref. 8) have developed "quadratic programming" and 
applied it to setting overall production levels for cases in which the cost 
functions are quadratic. 

For excellent summaries of the great amount of pertinent research and 
application in the inventory area, see Whitin (Refs. 33 and 34) and Simon 
and Holt (Ref. 35). See also Ref. 1, Chaps. 8-10. 

4. ALLOCATION MODELS 

Types of Problems. Allocation models are used to solve a class of prob- 
lems which arise when (a) a number of activities are to be performed and 
there are alternative ways of doing them, and (b) resources or facilities are 
not available for performing each activity in the most effective way. The 
problem is to combine activities and resources in such a way as to 
maximize overall effectiveness. These problems are divisible into two 
types: 

1. An amount of work to be done is specified. Certain resources are 
available; i.e., a fixed capacity and/or material for doing the job is avail- 
able and, hence, constitutes a restriction or limitation. The problem is to 
use these limited facilities and/or materials to accomplish the required work 
in the most economical manner. 

2. The facilities and/or materials which are to be used are considered to 
be fixed. The problem is to determine what work, if performed, will yield 
the maximum return on use of the facilities and/or materials. 

Linear Programming. Generally speaking, linear programming tech- 
niques can be used to solve a special class of allocation problems for which 
the following conditions are satisfied : 

1. There must exist an objective, such as profit, cost, or quantities, which 
is to be optimized and which can be expressed as, or represented by, a 
linear function. 

2. There must be restrictions on the amount or extent of attainment of 
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the objective and these restrictions must be expressable as, or represent- 
able by, a system of linear equalities or inequalities. 

The general linear programming problem may be expressed mathemati- 
cally as follows: 

Problem-Statement I. Find the values of X 1} X 2 , X 3 , • • •, X n which 
maximize (minimize) 

(52) Z = X l C l + X 2 C 2 + • • • + X n C n 
subject to the conditions that 

(53) Xy^O, i=l, 2, ...,n 
and 



(54) 



X\an -f- X 2 «i2 + • • • + X n ai n = &i 
X\a 2 i -T X 2 a 22 ~r " * ■ T~ X n a 2n = b 2 

X\a m \ -f- A2fl m 2 -f- ' • • "f" X n a mn = b m 



where a t -y, b;, and Cj are given constants (i = 1, 2, • • •, m; j = 1, 2, • • •, n). 
Problem-Statement II. Given the column vectors from eq. (54), 



Pj = 



(55) 



Pa = 



aij 

«2i 



Mm] 
&2 



3 TO . 



i = i, 2, 



the problem can also be stated as follows: Determine non-negative values of 
Xi, X 2 , • • •, X n which maximize (minimize) the linear functional 



(52a) Z = Xid + X 2 d + ■ • • + X n C n = £ XjPj = P . 
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Solution of Linear Programming Problems 

Among the several techniques which can be used to solve linear program- 
ming problems, the most important ones are the simplex technique and the 
transportation technique. There is also a special linear programming prob- 
lem, called the assignment problem, for which special techniques greatly re- 
duce the tremendous amount of computation that would otherwise follow 
from the use of either the transportation or simplex techniques. The assign- 
ment problem is discussed in Ref. 1, Chap. 12. 

Solution of Linear Programming Problems by the 
Simplex Technique 

The solution of linear programming problems by the simplex technique 
may best be illustrated by the solution of a specific problem. The prob- 
lem, simplified for purposes of illustration, may be stated as follows. 

Problem. A manufacturer Avishes to maximize the profits associated 
with producing two products, R and S. Products R and S are manufac- 
tured by a two-stage process in which all initial operations are performed 
in machine center I and all final operations may be performed in either 
machine center IIA or in machine center IIB. Machine centers IIA and 
IIB are different from each other in the sense that, in general, for any given 
product they yield different unit rates and different unit profits. In addi- 
tion, a certain amount of overtime has been made available in machine 
center IIA for the manufacture of products R and S. Since the use of over- 
time results in changes (decreases) in unit profits (but not in unit rates), 
let us denote separately, by machine center IIAA, any overtime use of 
machine center IIA. 

The unit times required to manufacture products R and S, the hours 
available in each machine center, and the unit profits are given in Table 4. 
In this table, R\, R 2 , and R s denote the three possible combinations for 
producing R, and similarly, Si, S 2 , and S 3 are defined for product S. 

Table 4. Unit Times Required to Manufacture Products R and S 

Product R Product S 



Machine 
Operation Center 


Ri 


R2 


R* 


Si 


s 2 


S3 


Hours 
Available 


1 I 


0.01 


0.01 


0.01 


0.03 


0.03 


0.03 


850 


2 IIA 
IIAA 
IIB 


0.02 


0.02 


0.03 


0.05 


0.05 


0.08 


700 
100 
900 


Profit per part 
(in dollars) 


0.40 


0.28 


0.32 


0.72 


0.64 


0.60 
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The problem is to determine how much of each product should be made 
through the use of each possible combination of machine centers so as to 
maximize the total profits, and to keep in mind the prescribed limitations 
on the capacities of the machine centers. The assumption here is that one 
can sell all that one can produce. This is a simplification which may be 
removed very easily by imposing additional restrictions in the form of 
maximum permissible quantities of each product. (See Ref. 36.) 

Simplex Solution. The simplex technique is a procedure which, 
through a series of repetitive arithmetic operations, progressively ap- 
proaches, and ultimately reaches, an optimum solution. The procedure 
may be summarized briefly as follows: 

1. The problem is first set up in mathematical form in which all relevant 
initial relationships and restrictions are stated. 

2. The problem is then set up in tabular form. 

3. An initial (feasible) solution is determined. 

4. Alternative changes to this solution are evaluated. 

5. A new solution is determined by introducing the "most favorable" 
alternative change. 

6. Steps 4 and 5 are repeated to derive successively better solutions. 

7. When, at any stage, step 4 evaluates no alternative choice favorably, 
the procedure is complete and gives an optimal solution. 

More explicitly, the simplex technique is carried out as indicated in the 
following steps: 

Step 1. Rephrase the problem in mathematical form. Let X x , X 2 , X 3 , 
X 4 , X 5 , Xq denote the amounts to be made of products Ri, R 2 , R 3 , Si, $ 2 > 
S 3 , respectively. Then the total profit Z will be given by 

(56) Z = 0.40X! + 0.28X 2 4- 0.32X 3 + 0.72X 4 + 0.64X 5 4- 0.60X 6 . 

Furthermore, the restrictions to the problem will be given by 

O.OlXi 4- 0.01X 2 + O.OIX3 + 0.03X4 + 0.03X5 + 0.03X 6 ^ 850 

0.02Xi + O.O5X4 < 700 
(57) 

0.02X2 + O.O5X5 ^ 100 

0.03X3 + 0.08X 6 ^ 900. 

Therefore, the problem may now be restated as follows : Determine the val- 
ues of Xj ^ (where j = 1, 2, • • •, 6) which maximize eq. (56) subject to 
the restrictions of eqs. (57). 

The restrictions Xj ^ 0, j = 1, 2, • • •, 6, arise from the fact that, since 
the manufacturing process is irreversible, one must preclude the appear- 
ance of negative values for these variables. 
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Step 2. Reduce the system of inequations (i.e., the restrictions) to an 
equivalent system of equations by introducing new non-negative variables 
X 7 , X 8 , X Q , X 10 . These new variables, X 7 , X 8 , X 9 , and X 10 , are variously 
called "disposal activities," "pseudo variables," or "slack variables." In 
this problem, it can be seen that positive values of these slack variables 
represent underutilization of capacity in machine centers I, IIA, IIAA and 
IIB respectively. The introduction of these slack variables results into the 
system of equations: 

O.OlXi + O.OIX2 + O.OIZ3 + O.O3X4 + 0.03X 5 + 0.03X G + X 7 = 850 

0.02X! + O.O5X4 + X 8 = 700 
(58) 

0.02X 2 4- O.O5X5 + X 9 = 100 

O.O3X3 + 0.08X 6 + X 10 = 900. 

Step 3. Complete the transformation of the given set of eqs. (56) and 
(58) into the standard form used in the simplex technique by making the 
following set of transformations. Rearrange eqs. (58) so that correspond- 
ing X/s appear in the same column. Then let the symbol Pj denote the 
column of coefficients of Xy (j = 1, 2, • • •, 10), and P denote the right- 
hand column of numbers in the system of eqs. (58). 

Assuming a zero profit or cost associated with each slack variable X 7 , 
Xg, X 9 , X10, the linear programming example may now be restated as fol- 
lows: Determine the values of a set of non-negative Xy (where j = 1, 2, 
• • • , 10) which maximize the linear form (functional) 

(56a) Z = 0.40X! + 0.28X 2 + 0.32X 3 4- 0.72X 4 + 0.64X 5 

+ 0.60X 6 + 0-X 7 + 0-X 8 4- 0-X 9 + 0-X 10 

subject to the restrictions 

10 
(58a) £ XyPy = P . 

Step 4- Exhibit the column vectors Pj in a systematic, i.e., tabular, form. 
This is done in Table 5 by means of eqs. (58), all blank spaces in the table 
representing zeros. 

It should be noted that eqs. (58) can be generated simply by multiplying 
each coefficient in any Pj column by the corresponding Xy and then reading 
across the rows. (The bold vertical line shows where to place the equal 
signs.) 

The square submatrix formed by {P 7) P 8 , P g , Pi }, which consists of 
elements that are equal to 1 on the main diagonal and that are everywhere 
else equal to zero, is of special importance.- This matrix is called the unit 
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Table 5. Column Vectors for Simplex Solution 



Pi 


P 2 


P S 


Pi 


P* 


Pe 


Pi 


P 8 


P 9 


Pio 


Po 


0.01 


0.01 


0.01 


0.03 


0.03 


0.03 


1 








850 


0.02 






0.05 








1 






700 




0.02 






0.05 








. 1 




100 






0.03 






0.08 








1 


900 



or identity matrix. The set of vectors which form the identity matrix are, 
in turn, said to be a unit basis of the particular space of interest, which is, 
in this problem, a four-dimensional space. The basis vectors are linearly 
independent vectors in terms of which every point in the ?i-dimensional 
(here, n = 4) space may be uniquely expressed and in terms of which a 
solution (or solutions) will be stated. 

Step 5. The columns of Table 5 are now rearranged as shown in Table 6a. 
Then, a column labeled "Basis" is inserted to the left of the P column and, 
in this column, the basis vectors are listed. For this example, the slack 
vectors form the unit basis. In some problems for which some of the re- 
strictions are stated either in terms of equalities or in terms of inequalities 
which impose minimum limits, so-called artificial vectors will have to be 
introduced in order to form a unit basis (see Ref. 36). It should be noted 
also that structural vectors may be such that they may be included in the 
unit basis. 

Next, a row of C/s is added, where the C/s are defined as the coefficients 
of the corresponding X/s in the expression for Z given in eq. (56). Then, 
a column of C/s is added, these corresponding to the C/s, but having the 
subscript i to denote the row, rather than the subscript j, which is used to 
denote the column. The expression for Z can now be written as 



(59) 



10 

y=i 



Step 6. Next, add a row of numbers labeled Zj, where j denotes the ap- 
propriate column. Letting X^ denote the element in the ith. row and jth. 
column of the table, the Z/s (including Z ) are defined by 



(60) 



Lj — / j L>i2i.ij. 



Table 6. Simplex Method (Ref. 1) 
(a) First Feasible Solution 



— 














0.40 


0.28 


0.32 


0.72 


0.64 


0.60 




Basis 


Po 


Pi 


P 8 


P 9 


Pio 


Pi 


Pi 


P 3 


Pa 


P 5 


P 6 




Pi 


850 


1 








0.01 


0.01 


0.01 


0.03 


0.03 


0.03 




Pa 


700 


. . _ 


1 






_ 0.02 






|0.05| 








P 9 


100 






1 






0.02 






0.05 






Pio 


900 








1 






0.03 






0.08 




Zi 


























Zi - ^ 












-0.40 


-0.28 


-0.32 


-0.72* 


-0.64 


-0.60 













(b) Second Feasible Solution 












Pi 


430 


1 


-0.6 






-0.002 


0.01 


0.01 




0.03 


0.03 


0.72 


Pa 


14,000 




20 






0.4 






1 








P 9 


100 






1 






0.02 






|0.05| 






Pio 


900 








1 






0.03 






0.08 




Zj 


10,080 




14.4 






0.288 






0.72 








Zi - Ci 


10,080 




14.4 






-0.112 


-0.28 


-0.32 




-0.64* 


-0.60 



m 

TO 

> 

O 
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TO 

O 

n: 






Table 6. Simplex Method (Ref.- 1)— Continued 
(c) Third Feasible Solution 





^s 














0.40 


0.28 


0.32 


0.72 


0.64 


0.60: 






Basis 


Po 


Pi 


P 8 


P 9 


Pio 


Pi 


P 2 


Pz 


Pi 


Ps 


-Pi 






Ft 


370 


1 


-0.6 


-0.6 




-0.002 


-0.002 


0.01 






0.03 




0.72 


Pa 


14,000 




20 






0.4 






1 






> 


0.64 


Ps 


2,000 






20 






0.4 






1 




«- 




Pio 


900 








1 






0.03 






|0.08j 


































Zj - Ci 


11,360 




14.4 


12.8 




-0.112 


-0.024 


-0.32 






-0.60* 



tn 

CO 
00 



(d) Fourth Feasible Solution 





Pi 


32.5 


1 


-0.6 


-0.6 


3 

— 8 


-0.002 


-0.002 


-1 
800 








-0.72 


Pi 


14,000 




20 






|0-4| 






1 






0.64 


Ps 


2,000 






20 






0.4 






1 




0.60 


Pe 


11.250 








12| 






3 

8 






1 






























Z } - Cj 


18,110 




14.4 


12.8 


7i 

'2 


-0.112* 


-0.024 


-0.095 









(e) Fifth Feasible Solution 



>5 














0.40 


0.28 


0.32 


0.72 


0.64 


0.60 




Basis 


Po 


Pi 


Fs 


P 9 


Pio 


Pi 


P 2 


Pz 


Pa 


Pt, 


Pt 




Pi 


102.5 


1 


i 

~2 


-0.6 


-f 




-0.002 


""TOTT 


0.005 






0.40 


Pi 


35,000 




50 






1 






* 






0.64 


Pe 


2,000 






20 




- 


0.4 






1 




-0.60 


i>6 


11,250 








100 






1 






1 






























Zi - d 


22,030 




20 


12.8 


7* 




-0.024 


-0.095* 


0.28 






(/) Sixth Feasible Solution 




Pi 


140 


1 


i 

— 7 


-0.6 


3 




-0.002 




0.005 




366' 


0.40 


Pi 


35,000 




50 






1 






* 






-0.64 


Ps 


2,000 






20 






|0.4| 






1 




0.32 


Pz 


30,000 








100 






1 






* 






























Zi - Cj 


24,880 




20 


12.8 


¥ 




-0.024* 




0.28 




0.25£ 
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TO 

> 

o 

z 

en 



CO 

m 
> 

TO 

n 
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Table 6. Simplex Method (Ref. 1) — Continued 
(g) Maximum Feasible Solution 



1 ^ 














0.40 


0.28 


0.32 


0.72 


0.64 


0.60 




Basis 


Po 


Pi 


Ps 


P* 


Pio 


Pi 


P 2 


Ps 


Pt 


Pi 


P* 




Pi 


150 


1 


i 

~~ 2 


i 

~2 


i 

— 3 








i 
Too' 


i 

Too" 


i 
Wo" 


.0.40 


Pi 


35,000 




50 






1 






5 
2 






0.28 


Pi 


5,000 






50 






1 






5 
2 




0.32 


Pz 


30,000 








100 
3 






1 






8 
7 






























Zj — Cj 


25,000 




20 


14 


iof 








0.28 


0.06 


0.25| 
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Step 7. A row labeled Zj — Cj is entered into the table and for any 
column, say jo, consists of the corresponding Cy subtracted from the value 
of Zjo which was entered in the previous row. 

Steps 1 through 7 complete the first phase of the simplex technique calcu- 
lations and result in what is known as a feasible solution to the problem, 
namely a solution which satisfies all the restrictions but which does not 
necessarily yield the optimum result. This feasible solution is given by the 
column vector P (Table 6a) in terms of the basis vectors Py, Pg, -Pg, Pw, 
namely, 

(61) X 7 = 850; X 8 = 700; X 9 = 100; X 10 = 900. 

That is, the initial feasible program consists of "Do not use any of the time 
available in any of the machine centers; i.e., do nothing," thus resulting in 
a net profit of Z = 0. 

Optimum Solution Criteria. Having obtained a feasible solution, 
one can proceed to the optimum solution by considering the following mutu- 
ally exclusive and collectively exhaustive possibilities: 

Ml. Maximum Z = 00 (i.e., maximum Z is infinitely large) and has been 
obtained by means of the present program. 

M2. Maximum Z is finite and has been obtained by means of the pres- 
ent program. 

M3. An optimum program has not yet been achieved and a higher value 
of Z may be possible. 

The simplex technique is such that possibilities Ml or M2 must be 
reached in a finite number of steps. Furthermore, if one remembers that 
Xij denotes the element in the ith. row and jth column of the table, the 
technique is such that, for a given tableau (i.e., table or matrix) : 

CI. If there exist any Zj - Cj < 0, either Ml or M3 holds: (a) if all 
X^ ^ in that column (for which Zj — Cj < 0), then Ml is true; (6) if 
some X{j > 0, further calculations are required, i.e., M3 holds. 

C2. If all Zj — Cj ^ 0, a maximal Z has been obtained (M2). 

Iterative Procedure to an Optimum Solution. In the example 
(Table 6a), Z± — C\ < (as are Z 2 — C 2 through Z 6 — Cq) and, further- 
more, some of the coefficients under Pi are greater than zero. Hence, by 
condition C16, further calculations are required (i.e., condition M3 
holds). 

To discover new solutions, it is possible to proceed in a purely systematic 
fashion by the simplex technique. Furthermore, any new solution so ob- 
tained will never decrease the value of the objective functional (although 
an increase need not occur), and, as stated earlier, the optimal solution, if 
one exists, must be reached in a finite number of steps. Hence, the simplex 
technique is a converging iterative procedure. 
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Step 8. Of all the Zj — Cj < 0, choose the most negative. (In the par- 
ticular example, this is Z 4 — C 4 = —0.72 and is so indicated by an asterisk 
in Table 6a.) This determines a particular Pj (namely P 4 ) which will be 
introduced into the column labeled "Basis" in Table 6&. 

Step 9. Determine the vector which this Pj will replace by dividing all 
the positive Xij appearing in the Pj column into the corresponding X^ 
which appears in the same row under P . (Since all the components of Po 
must be non-negative, all these ratios must, in turn, be non-negative.) The 
smallest of these ratios then determines the vector to be replaced. In the 
present example, P 4 is to replace one of the vectors P7, P 8 , P 9 , or P 10 . 
Under P 4 , there are two positive X^, namely X 7)4 = 0.03 and X 8)4 = 0.05. 
The division of these Xj/s into the corresponding X i0 's which appear under 
P gives a minimum of 14,000 (i.e., 700/0.05). Thus, P 8 is the vector to 
be replaced by P 4 , so that a new basis is formed consisting of the vectors 
P 7 , P 4 , P 9 , and P 10 (see Table 6&). 

Step 10. Let subscript k denote "coming in," subscript r denote "going 
out," X'ij denote the elements of the new matrix, and 

(62) <}> = min . X ik > 0, 

[i.e., <j> is the minimum of all ratios (Xi /Xi k ) for X ik > 0]. The elements 
of the new matrix (X'ij) are calculated as follows. The elements, X'kj, of 
the row corresponding to the vector just entered into the unit basis are 
calculated by 

(63) X' kj = |^. 

The other elements (X'ij) of the new matrix are calculated by 



\X r k/ 



(64) X'ij = X^ -\—^)X ik , 

\X r h/ 

where eq. (64) also applies to the Xj 's appearing under P and to the Zj 
— Cj in the entire bottom row (but not to the Z/s in the second to the 
last row). 

The new value of the profit function will be given by 

(65) Z' = Z - <j>(Z k - C k ), 
or, since Co = 0, the profit function will be given by 

(66) (Z - Co)' = (Zo ~ Co) - 4>(Zk - C k ). 



OPERATIONS RESEARCH 15-43 

For example, starting with Table 6a and proceeding to Table 6b, the 
most negative Zj — Cj is Z 4 — C 4 = —0.72. Therefore k = 4. Hence, 
from eq. (62), 

X jo 

$ = min for all X; 4 > 0, 

i X{4 



i.e., 



/850 700 \ 

<£ = min ( = 28,333; = 14,000 J = 14,000. 

\0.03 0.05 / 



Therefore, P 4 will replace P 8 ; or, in our notation, k = 4, r = 8. 

The elements in the P 4 row of Table 66 are then computed by eq. (63) 



-^81° 

Zi OJ 
4} — 

-^84 



Vo.05/ 
Therefore, 

/Xso\ _ ( 700 \ _ 



^0.05/ V 



X'4o = (— = — - = 14,000, 



X 41 — 



Vo.05/ Vo.05/ 

-^81 

0.05 



\ /0.02\ 

I = ) = 0.4, etc. 

/ Vo.05/ 



For the elements of the other rows, where k = 4, r = 8 are substituted 
into eq. (64), 

x '" = x,i ~ (S) Xu = x " " fcl) Xii - 

Therefore, 

/^8o\ /700\ 

X' 70 = X 70 - 1 (X 74 ) = 850 - ) (0.03) 

VO.05/ Vo.05/ 

= 850 - (14,000) (0.03) = 850 - 420 = 430. 
and 



(z x - c x y = (Zx - co - (^) (z 4 - c 4 ) 



/0.02\ 

= (-0.40) - H-0.72) 

V0.05/ 

= (-0.4) - (0.4) (-0.72) = -0.4 + 0.288 = -0.112, etc. 
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Finally, the new value of the profit functional will be given (see Table 66) 



by 



(Z — Co)' = (Z — Co) — <£(Z 4 ~~ C4) 

= - 14,000(-0.72) = +10,080. 



The results are shown in Table 66. 

Step 11. The process is then repeated until such time as either condition 
Ml or condition M2 holds. For the present example, the solution is ob- 
tained after six iterations, i.e., six tableaux or matrices after the first (see 
Tables Qa-g). The final tableau, Table Qg, yields the optimum solution. 
(If any other optimum solutions existed, they would be indicated by Zj 
— Cj = for j's other than those appearing in the basis. Here, Zj — Cj 
= for j — 1, 2, 3, and 7 only. Hence no other optimum solutions exist.) 
This optimum solution is also stated, both in terms of the number of parts 
and hours required, in Tables 7 and 8. 



Table 7. Optimum Program (Number of Parts) 

Product R Product S 



#1 (Centers I-IIA) 
R 2 (I-IIAA) 
Rz (I-IIB) 
Total 

Total profit 



35,000 parts 

5,000 parts 

30,000 parts 

R = 70,000 parts 



-Si 
S3 



parts 
parts 
parts 

S = 



$25,000 + = $25,000 







Pi 


roduct 


,R 


Product S 






Sur- 


Opera- 


Machine 










Hours 


Hours 


plus 










tion 


Center 


Ri 


R2 


Rz 


Si $2 $3 


Used 


Avail. 


Hours 


1 


I 


350 


50 


300 





700 


850 


150 


2 


IIA 


700 









700 


700 







IIAA 




100 







100 


100 







IIB 






900 





900 


900 






Thus, one readily sees that the optimum (most profitable) program under 
the prescribed conditions consists of manufacturing 70,000 units of prod- 
uct R to the complete exclusion of product S. Furthermore, by eq. (56) 
and also by (Z — C ) in the optimum tableau, the total profits will be 

Z = 0.40(35,000) + 0.28(5,000) + 0.32(30,000) + 0.72(0) 

+ 0.64(0) + 0.60(0) 

= $25,000. 
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Alternate Step 8. One should note at this point that the improvement 
from one tableau to the next is given by —cf>(Zk — Cu), see eq. (65). Fur- 
thermore, in practice, one need not select the most negative number (Zj — Cj) 
but, rather, that negative number which yields the greatest improvement. Thus, 
in the example, 

/700\ 
-*{Z X - Ci) = - {—) (-0.40) = 14,000, k = 1 

/100\ 
-4>(Z 2 ~ C 2 ) = - ^— J (-0.28) = 1,400, h = 2 

/900\ 
-4>(Z Z - C,) = - {—) (-0.32) = 9,600, k = 3 

/700\ 
-*(Z 4 - C 4 ) = - ^— j (-0.72) = 10,080, ft = 4 

/100\ 
-<A(Z 5 - C 5 ) = - ^— J (-0.64) = 1,280, k = 5 

/900\ 
-c^(Z 6 - C 6 ) = - ^— J (-0.60) = 6,750, fc = 6. 

Therefore, instead of introducing P 4 into the basis, a greater gain is achieved 
at this step through the introduction of Pi. In this particular example, 
following alternate Step 8 enables one to reach the optimum solution, Table 
6g, in three less iterations. 

Further Restrictions in Linear Programming Problems. Once 
having established the solution to a given linear programming problem, one 
may wish to consider (or evaluate) further restrictions on the variables. 
Thus, by referring to the preceding example, these restrictions may be in 
the form of: (1) minimum requirements for product S, (2) changes in the 
amount of time available in the machine centers, (3) changes in the prices 
of the various products, (4) changes in the unit production rates, e.g., due 
to the "introduction" of new equipment. 

The simplex technique is such that, in general, new optimum solutions 
can easily be constructed in terms of such added restrictions by making use 
of the optimum solution to the original problem. For a full discussion of 
this point, see Ref. 1, Chap. 11. 

Solution of Minimization Problems by the Simplex Technique. 
To solve minimization problems by the simplex technique one may, in 
Step 8, select either (1) the most positive Z } - — Cj, or alternately (2) the 



15-46 OPERATIONS RESEARCH 

most negative Cj — Zj, and then proceed as before to the solution of the 
problem. 

The Transportation Problem 

A linear programming problem for which a special technique has been 
developed is the so-called transportation problem which may be stated as 
follows. 

Problem. Determine Xa ^ which minimize 

to n 

(67) Z=SE CijXa, 
such that 

n 

(68) HX tj = Ai (i = 1, 2, ..-,m) 
and 

TO 

(69) T,X ij = B j (j= 1, 2, ...,n). 

The transportation problem is obviously a special case of the general 
linear programming problem; hence, it can be solved by the simplex 
technique. 

However, a special solution technique, far simpler than the simplex 
technique, has been developed for solving transportation problems and, 
quite appropriately, it is called the transportation technique (Ref. 39). 

The procedure in the transportation technique is outlined as follows : 

1. The problem is set up in tabular form. 

a. All requirements are explicitly stated. 

b. All permissible slack in the system is explicitly stated. 

c. All appropriate costs and/or revenues are determined. 

d. An objective function is determined. 

e. The computational framework is established. 

2. An initial solution is determined. 

a. The initial solution must be technically feasible, i.e., it must meet all 
restrictions. 

3. Alternative choices are evaluated. 

a. Changes in the solution are made one at a time. 

b. The evaluation is of the complete effect of each unit change. 

4. The "most favorable" alternative is selected. 

5. The number of units to be included in this change is determined. 

a. Owing to the linear nature of the model, each unit contributes the 
same cost or profit difference. 
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b. The limit on the number of units involved in the particular change is 
technical feasibility (non-negativity requirements). 

6. A new solution is determined. 

a. The elements to change and the number of units to include have been 
previously determined. 

7. Steps 3 through 6 are repeated. The process is a converging iterative 
one. 

8. When Step 3 evaluates no alternative favorably, the procedure is com- 
plete and one has an optimal solution. 

Example. This example, taken from Ref. 37a, deals with the problem 
of moving empty freight cars from three "excess" origins to five "defi- 
ciency" destinations in such a manner that, subject to the given restric- 
tions, the total cost of the required movement will be a minimum. The 
specific conditions of the problem and the unit (per freight car) shipping 
costs are given in Tables 9 and 10. 

Table 9 states that origins Si, S 2 , and S 3 have surpluses of 9, 4, and 8 
empty freight cars, respectively, while destinations D{, D 2 , D 3 , Z> 4 , and D 5 

Table 9. Physical Program Requirements 



n^ Destina- 














\ tions 














\ 


Di 


D 2 


D 3 


Dt 


D h 


Surpluses 


Origins \^ 














Si 


In 


X12 


X\3 


X u 


x 15 


9 


s 2 


X21 


X22 


A 23 


X24 


X25 


4 


s s 


-^"31 


X32 


Xss 


X34 


-X35 


8 


Deficiencies 


3 


5 


4 


6 


3 


21 



are in need of 3, 5, 4, 6, and 3 cars, respectively. For simplicity, it has been 
assumed that the problem is self-contained, i.e., that the number of excess 
cars is equal to the number of deficiencies. Any transportation problem 
can be made self-contained through the introduction of dummy origins or 
destinations. 

Table 10 lists the unit costs C;y of sending an empty freight car from the 
ith origin to the jth destination. 
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Table 10. Unit Shipping Costs 



N. Destina- 
\. tions 

Origins \. 


Di 


D 2 


D s 


D A 


D 6 


Si 


-10 


C12 

-20 


-5 


Cu 

-9 


C15 

-10 


s 2 


C21 

-2 


C22 

-10 


C23 

-8 


C24 

-30 


C25 

-6 


S3 


C31 

-1 


C32 

-20 


C33 

-7 


C34 

-10 


C35 

-4 



The solution to this problem by the transportation technique is obtained 
as indicated in the following steps. 

Step 1. Set up the tables listing the physical program requirements 
(Table 9) and unit shipping costs (Table 10). 

Step 2. Obtaining a First Feasible Solution. Write down an initial (feas- 
ible) solution, namely one which satisfies the movement requirements. (If 
a feasible solution also minimizes the total cost, it is then called an optimum 
feasible or, in this case, a minimal feasible solution). This can easily be done 
by applying a technique which has been developed by Dantzig, Ref . 38, and 
which Charnes and Cooper, Ref. 39, refer to as "the northwest corner rule*" 

The northwest corner rule may be stated as follows : 

1. Start in the upper left-hand corner of Table 9 (requirements) and 
compare the amount available at S\ with the amount required at D\. (a) 
If Di < Si, i.e., if the amount needed at Z>i is less than the number of units 
available at Si, set I n equal to Di and proceed to cell X 12 , i.e., proceed 
horizontally. (6) If Di = Si, set X n equal to Di and proceed to cell X 2 2, 
i.e., proceed diagonally, (c) If Di > Si, set Xn equal to Si and proceed 
to X 2 i, i.e., proceed vertically. 

2. Continue in this manner, step by step, away from the upper left cor- 
ner until, finally, a value is reached in the lower right corner. Thus, in the 
present example (see Table 11), proceed as follows: 

(a) Set Xn equal to 3, namely, the smaller of the amount available at 
Si (9) and that needed at Di (3). 



OPERATIONS RESEARCH 



15-49 



Table 11. First Feasible Solution 



\^ Destina- 
\^ tions 

Origins N. 


Di 


D 2 


D 3 


Di 


D 6 


Total 
Surpluses 


Si 


© 


© 


® 






9 


s 2 






© 


© 




4 


s 3 








© 


© 


8 


Total defi- 
ciencies 


3 


5 


4 


6 


3 


21 



(b) Proceed to X 12 (rule la). Compare the number of units still avail- 
able at $i (namely 6) with the amount required at D 2 (5) and, accordingly, 
letXia = 5. 

(c) Proceed to X 13 (rule la), where there is but one unit left at Si while 
four units are required at D 3 . Thus set X 13 = 1. 

(d) Then proceed to X 23 (rule lc). Here X 23 = 3. 

(e) Continue and set X 24 = 1, X 34 = 5, and, finally, in the southeast 
corner, set X35 = 3. 

The feasible solution obtained by this northwest corner rule is shown in 
Table 1 1 by the circled values of the X{j. That this set of values is a feas- 
ible solution is easily verified by checking the respective row and column 
requirements. The corresponding total cost of this solution is obtained by 
multiplying each circled X^ in Table 1 1 by its corresponding C# in Table 
10 and summing the products. For any cell in which no circled number 
appears, the corresponding X^ is equal to zero. That is, the total cost is 

given by: 

53 35 

(70) Total cost = S E di^a =EZ Ctf^tf- 



j=\ i=i 



i=i y=i 



The total cost associated with the first feasible solution is computed as 
follows: 



T.C. = X11C11 + X12C12 + X13C13 4- X23C23 + -X24C24 + X34C34 
+ X35C35 
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T.C. = (3)(-10) + (5) (-20) + (l)(-5) + (3)(-8) + (l)(-30) 
+ (5)(-10) + (3)(-4) 
= —$251 (minus sign means "cost" rather than "profit"). 

Step 3. Evaluation of Alternative Possibilities. Evaluate alternative pos- 
sibilities, i.e., evaluate the opportunity costs associated with not using the 
cells which do not contain circled numbers. Such an evaluation is illus- 
trated by means of the program given in Table 1 1 and is exhibited in Table 
12 (noncircled numbers only). This evaluation is obtained as follows. (For 
an alternative method of evaluation, see Ref. 1.) 

Table 12. First Feasible Solution (with Evaluations): C = 251 



\. Destina- 
\^ tions 

Origins ^\ 


Di 


D 2 


D 3 


D 4 


D 6 


Total 


Si 


© 


© 


© 




-11 


9 


-18 




s 2 


-11 


-13 


© 


® 


-18 


4 


s 3 


8 


17 


19 


© 


© 


8 


Total 


3 


5 


4 


6 


3 


21 



1. For any cell in which no circled number appears, describe a path in 
this manner. Locate the nearest circled-number cell in the same row which 
is such that another circled value lies in the same column. 

Thus, in Table 12, if one starts with cell S 3 Di (row 3, column 1), the 
value © at S3D4 (row 3, column 4) satisfies this requirement; i.e., it is the 
closest circled-number cell in the third row which has another circled value, 
® at S2D4, in the same column (column 4). The circled number ® in 
position S 3 D 5 fails to meet this requirement. 

2. Make the horizontal and then the vertical moves so indicated. In 
the example, move from S 3 Di to S 3 D 4 (see Table 12). 

3. Having made the prescribed horizontal and vertical moves, repeat the 
procedure outlined in Steps 1 and 2. For the example, this now gives cells 
S 2 D 3 and SiD 3 respectively; accordingly, one moves from © at S2D4 to 
® at &1D3 by way of © at S%D 3 . 
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4. Continue in this manner, moving from one circled number to another 
by, first, a horizontal move and then a vertical move until, by only a hori- 
zontal move, that column is reached in which the cell being evaluated is 
located. (The fewest steps possible should be used in this circumambula- 
tory procedure.) Thus, to continue the example, this step is from (T) at 
SiD 3 to (?) at SiDi. 

5. Finally, move to the cell being evaluated (here, S 3 Di). This com- 
pletes the path necessary to evaluate the given cell: (Note. For the pur- 
poses of evaluation, the path ends, rather than starts, with the cell being 
evaluated.) 

6. Form the sum, with alternate plus and minus signs, of the unit costs 
associated with the cells being traversed. (These unit costs are given in 
Table 10.) This is the (noncircled) evaluation to be entered into the ap- 
propriate cell in Table 12. Thus, for the example, one has for the evalua- 
tion of cell *S 3 Di : 



Path (Table 12) 


SzDt 


S2D4 


S2D3 


S1D3 


S1D1 


S3D1 


Unit cost (Table 10) 


-10 


-30 


-8 


— 5 


-10 


-1 


Evaluation (S3D1) 


+ (-10) 


-(-30) 


+ (-8) 


-(-5) 


+ (-10) 


-(-1) 



+8 

Accordingly, one enters +8 in cell S3D1 of Table 12. 

7. Repeat the procedure outlined until all cells not containing circled 
numbers are evaluated. 

Step 4- Iterative Procedure toward an Optimum Solution. If the noncir- 
cled numbers (the evaluations) are all non-negative, an optimum has been 
achieved. If one or more noncircled numbers are negative, further im- 
provement with respect to the objective function is possible (e.g., the nega- 
tive numbers in S1D4, S 2 D 2 , etc., in Table 12). (At this stage, it should be 
quite apparent that one must be careful to circle the values of X{j obtained 
in a feasible solution in order to distinguish them from the "evaluation" 
numbers which are also in the same table.) 

Improvement is obtained by an iterative procedure in which one pro- 
ceeds as follows: 

(a) Of the one or more negative values which appear, select the most 
negative one, say —N. If there are more than one such values, any one of 
these may be selected arbitrarily. 

(6) Retrace the path used to obtain this most negative value. 

(c) Select those circled values which were preceded by a plus sign in the 
alternation between plus and minus and, of these, choose the one with the 
smallest value written in its circle, say m. 

(d) One is now ready to form a new table, wherein one replaces the most 
negative value, —N, by this smallest value, m. 

(e) Circle the number m and then enter all the other circles (except the 
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one which contained the value m in the previous program) in their previous 
cells, but without any numbers inside. 

The improvement in cost from one program to the next will then be 
equal to mN. Furthermore, as with the simplex technique, one need not 
select the most negative number. It is permissible, and sometimes advan- 
tageous, to select the first negative number which appears. Since the im- 
provement from one program to the next is given by mN, a study of Table 
12 shows that selections of S 2 D!, S 2 D 5 , S 2 D 5 , or SiD 5 would have resulted 
in improvements of 33, 39, 18, and 11 respectively, as compared with the 

Table 13. Iterative Procedure toward Solution 
(a) Value to Be Moved 



\^ Destina- 
\^ tions 

Origins \ 

\ 


Bi 


D 2 


D 3 


Di 


D 6 


Total 


Si 


O 


O 


O 


© 




9 


s 2 






O 






4 


$3 








O 


O 


8 


Total 


3 


5 


4 


6 


3 


21 



(6) Second Feasible Solution: C = 233 



\Destina- 
\. tions 

Origins \. 


Di 


D 2 


D 3 


Di 


D 5 


Total 


Si 





© 


® 


© 


7 


9 


s 2 


-li 


|-13| 


© 


18 





4 


s s 


-10 


-1 


1 


© 


© 


8 


Total 


3 


5 


4 


6 


3 


21 
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improvement of 18 resulting from the selection of S1D4. Another alterna- 
tive is to examine all products, mN, and select that negative numbered 
cell which results in the greatest improvement, in this case, $ 2 -D 5 . 

Thus in Table 12 the most negative number is — 18 and appears in both 
cells S1D4 and S 2 D 5 (i.e., — N = — 18). For such ties, one may arbitrarily 
select either of the cells containing this most negative number. Here, cell 
S1D4 is chosen. Retracing the path used to obtain the — 18 value in cell 
S1D4, one then obtains -\-S1D3, —S 2 D 3 , -\-S2D4, —S1D4. Of those pre- 
ceded by a plus sign, namely SxD 3 and S 2 D 4 , both have the circled value 
© in their cells. Consequently, either one of these may be chosen as the 
circled value to be moved. In this case, cell S 2 D4 is arbitrarily chosen. 
The circled value © is then entered into cell >SiD 4 (see Table 13a, i.e., that 
cell where — 18 appeared in Table 12). (Therefore, the improvement over 
the program given in Table 12 will be 1 X 18 = 18 cost units. That is, 
the next program (Table 136) will cost 251 - 18 = 233 cost units.) The 
other circles (without numbers) are then entered in the same positions as 
before (see Table 13a). 

Step 5. A new feasible solution is obtained by filling in the circles accord- 
ing to the given surplus-deficiency (input-output) specifications. This 
solution is given by the circled values in Table 13&. 

Step 6. The program is then evaluated, as before, and negative (non- 
circled) numbers still appear. 

Step 7. The process is successively repeated (Tables 14, 15, and 16) 
until, finally, in Table 16 the evaluation of the corresponding program 
given therein results in all (noncircled) numbers being non-negative. An 
optimum feasible solution, or program, therefore, has been reached. 



Table 14. Third Feasible Solution: C = 181 



\. Destina- 
\^ tions 

Origins \^ 


D x 


D 2 


D 3 


Di 


£5 


Total 


£1 


© 


© 


© 


© 


7 


9 


S 2 


2 


© 


13 


31 


13 


4 


s 3 


-10 


-1 


1 


© 


© 


8 


Total 


3 


5 


4 


6 


3 


21 
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Table 15. Fourth Feasible Solution: C = 151 



\ Destina- 
\ tions 

Origins \^ 


Di 


D 2 


D a 


Di 


D 6 


Total 


Si 


10 


© 


© 


© 


7 


9 


s 2 


12 


© 


13 


31 


13 


4 


s 3 


© 


B 


1 


© 


© 


8 


Total 


3 


5 


4 


6 


3 


21 



Table 16. Optimum Feasible Solution: C = 150 



\ Destina- 
\ tions 

Origins \^ 


Di 


D 2 


D 3 


Di 


D 6 


Total 


Si 


10 


1 


© 


© 


7 


9 


s 2 


11 


© 


12 


30 


12 


4 


Ss 


© 


© 


1 


© 


© 


8 


Total 


3 


5 


4 


6 


3 


21 



Alternate Optimum Programs. If any of the evaluation numbers in 
the optimum tableau are zero, alternate optimum tableaux exist. These 
alternate optimum solutions are obtained by essentially the same proce- 
dure as that which was just given. The only variation is that the zeros 
(if any) which appear in the optimum feasible solutions are now treated in 
exactly the same manner as were the negative values. 

Furthermore, given such alternate optimum programs, say {Pi}, {P2}, 
• • •, {P n }, where {P n \ refers to the set of Xij which form the nth optimum 
program, then 



(71) 



{P n+1 } =a 1 {P 1 } +a 2 {P 2 \ +...+ « n {P n } 
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is also an optimum program provided the a t - are non-negative constants 
such that 



(72) 



2 di = ai -f a 2 + a 3 -\ \- a n = 1. 



For example, the cost minimization problem represented by Table 17 has 
two optimum programs, namely those given in Tables 18 and 19. Table 19 
is obtained from Table 18 (and vice versa) by treating the zero in cell S 3 D 5 
of Table 18 (or cell S3D4 of Table 19) as the "most negative number" and 
proceeding as before. 

Table 17. Unit Cost Matrix 



\^ Destina- 
^^ tions 

Origins \. 


By 


D 2 


D 3 . 


D A 


£5 


Total 


-Si 


-2 


-1 


-4 


-3 





5 


£2 


+ 1 


-3 


-5 


-2 


-1 


7 


$3 


-1 


-4 


-3 


-2 


-1 


6 


Total 


2 


2 


5 


4 


5 


18 



Table 18. Optimum Program for Table 17 



\. Destina- 
\^ tions 

Origins \^ 


Dx 


D 2 


D 3 


Di 


D 6 


Total 


-Si 


4 


© 


2 


2 


© 


5 


-S2 


© 


1 


2 


© 


© 


7 


£3 


2 


2 


© 


© 





6 


Total 


2 


2 


5 


4 


5 


18 
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Table 19. Alternate Optimum Program for Table 17 



\Destina- 
^\ tions 

Origins >v 


D x 


D 2 


D s 


2>4 


2>6 


Total 


Si 


4 


© 


2 


2 


© 


5 


s 2 


© 


1 


2 


© 


© 


7 


S3 


2 


2 


© 





© 


6 


Total 


2 


2 


5 


4 


5 


18 



An infinite number of derived optimum programs can now be obtained 
by forming what are called "convex linear combinations" of the two basic 
optimum programs. Thus, if we select two positive fractions whose sum 
is unity, e.g., 34 and 24, we can obtain a new optimum program by multi- 
plying every element of the first program by 34 and every element of the 
second program by 24 and then adding corresponding cells. This yields 
the derived optimum program of Table 22 and is obtained as shown in 
Tables 20 and 21. Similarly, other optimum programs could be derived 
for other non-negative fractions whose sum is equal to 1. Note. In gen- 
eral, derived optimum programs will involve fractional answers. These 
programs are for use only where nonintegral answers are realistic. 





Table 20. 


i Table 18 = 






^^ Destina- 
n. tions 

Origins \^ 


Di 


D 2 


D z 


Di 


D„ 


Total 


Si 




® 






® 


5 


s 2 


® 






® 


® 


7 


S3 






® 


® 




6 


Total 


2 


2 


5 


4 


5 


18 
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\^ Destina- 
\v tions 

Origins \^ 


D x 


D 2 


Z>3 


D 4 


£s 


Total 


Si 




® 






® 


5 


s 2 


® 






© 


® 


7 


s 3 






(±]>) 




® 


6 


Total 


2 


2 


5 


4 


5 


18 



Table 22. A Derived Optimum Program: \ Table 20 + f Table 21 



\. Destina- 
\^ tions 

Origins N. 


B, 


D 2 


D 3 


Di 


2>6 


Total 


Si 




© 






© 


5 


s 2 


© 








® 


7 


s 3 






© 


© 


® 


6 


Total 


2 


2 


5 


4 


5 


18 



Solution of Maximization Problems by the Transportation Tech- 
nique. Although the exposition just given treats only a (linear) minimiza- 
tion problem, it should be obvious that the transportation technique is 
equally applicable to (linear) maximization problems. The only difference 
in solving maximization problems lies in the preparation of the "profit" 
matrix. Whereas in the minimization problem all costs are entered with a 
negative sign, here all profits (or whatever units are involved in the max- 
imization problem) are entered without any modification of signs. Once 
the initial datum matrix is obtained, one proceeds to the solution exactly 
as previously outlined. 



15-58 



OPERATIONS RESEARCH 



Variations on the Transportation Technique. Many variations on 
the transportation technique are available for the solution of transporta- 
tion problems. One has already been cited with respect to the selection of 
the cell to be introduced into the new basis. A second variation, designed 
to decrease the number of iterations, involves a rearrangement of the cost 
matrix. By using the problem cited in Tables 9 and 10, this may be illus- 
trated as follows. 

1. Form a new matrix in which the first row and first column correspond 
to the cell yielding the least cost. In the example, this is & 3 Z>i. Enter the 
totals of 8 for S 3 and 3 for D 1 in the new matrix. Place the smallest of 
these two numbers in that cell, S3D1. I 



\ 


Dx 




•! 






Total 


s 3 


3 










8 






























Total 


3 













2. This satisfies the requirement for D ly but still leaves 5 units available 
at S 3 . Hence, select the next least unit cost which involves S 3 . In the 
example, this is —4 in cell S 3 D 5 . Therefore, list D 5 in the second column 
and enter the corresponding total (requirement) of 3. Compare the re- 
quirement of 3 units at D 5 with the remaining availability of 5 units at S s , 
and assign 3 units to cell $3-D 5 . 



\ 


D t 


D 5 








Total 


Ss 


3 


3 








8 






























Total 


3 


3 
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3. Since 2 units are still available at S 3 , select the third highest cost, 
namely —7 in aS 3 D 3 . Enter D 3 in the third column along with its total 
requirement of 4 units. Comparing the requirement of 4 units at D 3 with 
the remaining availability of 2 (8 — 3 — 3) units at S 3 , assign 2 units to 
cell S 3 D 3 , thereby using all available units at S 3 but leaving 2 units still to 
be assigned to D 3 . 



\ 


Di 


D 6 


D s 






Total 


s s 


3 


3 


2 






8 






























Total 


3 


3 


4 









4. Compare the costs associated with D 3 (C\ 3 = — 5 and C 23 = — 8) 
and select Si as the entry for the second row and, with it, enter the avail- 
ability at Si, namely 9. Compare this availability at Si (i.e., 9) with the 
remaining requirement at D 3 (i.e., 2 = 4 — 2) enter 2 units in cell $iD 3 , 
and thereby satisfy the requirement at D 3 . 



\ 


Di 


£>5 


D 3 






Total 


s 3 


3 


3 


2 






8 


Si 












9 
















Total 


3 


3 


4 
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5. By proceeding in this fashion, the following matrix is obtained: 



\ 


Di 


D 5 


D* 


#4. 


D 2 


Total 


03 


3 


3 


2 






8 


Si 






2 


6 


1 


9 


£2 










4 


4 


Total 


3 


3 


4 


6 


5 


21 



The cost for this initial feasible solution is given by 

3(-l) + 3(-4) 4- 2(-7) + 2(05) + 6(-9) + l(-20) + 4(-10), 

i.e., neglecting the minus sign which indicates cost, 

T.C. = $153, 

as compared with the first feasible solution of $251 obtained by the north- 
west corner rule (and with the optimum solution of $150). Such a re- 
shuffling of the cost matrix generally leads to a better (i.e., lower cost or 
higher profit) first feasible solution so that the optimum solution is usually 
reached after a smaller number of alterations. 

The reader should note that this first feasible solution costing $153 could 
have been obtained without reshuffling the matrix. One simply starts in 
the cell of lowest cost (here S3D1) and proceeds accordingly. 

For further details of the transportation technique, including a discus- 
sion of so-called degenerate cases, see Ref. 39. The mathematical deriva- 
tion of the transportation technique is given in Ref. 40, Chap. 23. 

Alternate Method of Evaluating Cells in Transportation Tech- 
nique. An alternate evaluation technique (or procedure) is presented by 
means of the problem represented by Tables 23 and 24, namely the unit 
cost table and the table listing the first feasible solution of the transporta- 
tion problem given earlier (see Tables 10 and 11). The evaluation tech- 
nique presented here is a variation of that originally designed by Dantzig 
in Koopmans (Ref. 40, Chap. XXI), and is part of the procedure described 
in Henderson and Schlaifer (Ref. 41). The discussion of determining the 
costs of deviating from the optimum solution is given in Ref. 41. The first 
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Table 23. Unit Shipping Costs 



\^ Destina- 












\^ tions 


Dt 


Di 


D 3 


£>4 


D, 


Origins \^ 












£1 


-10 


-20 


-5 


-9 


-10 


s 2 


-2 


-10 


-8 


-30 


-6 


S 3 


-1 


-20 


-7 


-10 


-4 



Table 24. First Feasible Solution 



\^ Destina- 
\^ tions 

Origins ^\ 


Di 


D 2 


D 3 


2>4 


D & 


Total 


-Si 


3 


5 


1 






9 


S 2 






3 


1 




4 


s 3 








5 


3 


8 


Total 


3 


5 


4 


6 


3 


21 



part of the evaluation procedure is to form a new table (Table 25) corre- 
sponding to Table 24, but listing the unit costs rather than the amounts to 
be shipped. These costs are given by the boldface numbers in Table 25. 

Add to Table 25 a column labeled "Row Values" and a row labeled 
"Column Values" and calculate these values as follows: 

1. Assign an arbitrary value to some one row or some one column. For 
purposes of illustration, let us assign the value to row S\. 

2. Next, for every cell in row Si which contains a circled number repre- 
senting part of the feasible solution, assign a corresponding column value 
(which may be positive, negative, or zero) which is such that the sum of the 
column value and row value is equal to the unit cost rate. 

More generally, if r t - is the row value of the ith row, Cj the column value 
of the jth column, and C t -y the unit cost for the cell in the ith. row and yth 
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Table 25. Unit Costs and Fictitious Costs Corresponding to 
First Feasible Solution 



\^ Destina- 
\^ tions 

Origins \^ 


Di 


D 2 


D 3 


D 4 


D 6 


Row 

Values 


•: S, 


-10 


-20 


-5 


-27 


-21 





S 2 '' 


.-13 


-23 


-8 


-30 


-24 


-3 


s 3 


7 


-3 


12 


-10 


-4 


17 


Column Values 


-10 


-20 


-5 


-27 


-21 





column which contains a circled number, then all row and column values 
are obtained by the equation 



(73) 



T{ + Cj — W/- 



Thus, by assuming r x = 0, it can be immediately determined from eq. (73) 
that 

Ci = — 10; c 2 = — 20; c 3 = — 5. 



3. Next, since c 3 = — 5 and C 23 — — 8, determine that r 2 = — 3. 

4. Since r 2 = — 3 and C 24 = —30, then c 4 = —27. 

5. From c 4 = —27 and C 34 = —10, then r 3 = +17 is obtained. 

6. Finally, for r 3 = +17 and C 35 = —4, c 5 = —21 is obtained. 

This procedure for assigning row and column values can be used for any 
solution-matrix which is nondegenerate, i.e., given a matrix of m rows and 
n columns, where the solution consists of exactly m + n — 1 nonzero ele- 
ments. (Any solution consisting of less than m + n — 1 nonzero elements 
is said to be degenerate. Simple methods for dealing with degeneracy may 
be found in Charnes and Cooper (Ref. 39), Henderson and Schlaifer (Ref. 
41), and Dantzig (Ref. 38).) 

After all row and column values for Table 25 have been computed, the 
table can be completed by filling in the remaining cells according to eq. 
(73). This results in the lightface figures given in Table 25. 

After Table 25 has been completed, the cell evaluations may be obtained 
as follows. Form a new table (Table 26) which consists of the unit cost 
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rates of Table 23 subtracted from the number in the corresponding cell of 
Table 25. That is, in symbolic notation, 

{Table 26} = {Table 25} - {Table 23}. 

The cells corresponding to movements which are part of the solution will 
contain zeros. These zeros are given in boldface type in Table 26. The 

Table 26. Cell Evaluations for the First Feasible Solution 



Ny Destina- 
\^ tions 

Origins \^ 


D x 


Do 


D 3 


D, 


£>5 


Si 











-18 


-11 


St 


-11 


-13 








-18 


Ss 


8 


17 


19 









resulting numbers for the remaining cells are given in lightface type and 
are the cell evaluations to be used in determining a better program or solu- 
tion. (Comparison with Table 12 will show this to be true.) 

When these cell evaluations have been determined, proceed as previously 
outlined in the section. 

Geometric Interpretation of the Linear Programming Problem 

A geometric interpretation of the linear programming problem may be 
given by means of the following specific two-dimensional example. 

Problem 1. To determine X, Y ^ which maximize Z = 2X + 5F 
subject to 

X <4, 



(74) 



Y ^ 3, 
X + 2Y ^ 8. 



The system of linear inequalities which constitute the restrictions results 
in the convex set of points given by polygon OABCD of Fig. 6. That is, 
any point (X, Y) on or within the polygon satisfies the entire system of in- 
equalities (74). Hence, there exist an infinite number of solutions to sys- 
tem (74). The linear programming problem then is to select, from this 



15-64 



OPERATIONS RESEARCH 



X = 4 




Y = 3 



O D 

Fig. 6. Region satisfying restrictions (23) for non-negative X and Y (Ref. J). 



2X+5Y = Z 



■X 



Fig. 7. Family of parallel straight lines, Z = 2X + 5Y (Ref. 1). 




2X+5Y=8 
2X+5y=0 



2X+5y=15 



Fig. 8. Figure for geometric solution of linear programming problem (Ref. 1). 
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infinite number of points, the one or more points which will maximize the 
function Z = 2X + 5Y. 

The function Z = 2X + 5F is a one-parameter family of parallel lines; 
i.e., the function represents a family of parallel straight lines (of slope 
— %) such that Z increases as the line gets farther removed from the ori- 
gin; see Fig. 7. The problem may then be thought of as one of determining 
that line of the family, 2X -f 5F = Z, which is farthest away from the 
origin but which still contains at least one point of the polygon OABCD. 

Figure 8 shows how several members of the family Z = 2X + 5F are 
related to the polygon OABCD and, in particular, shows that the solution 
is given by the coordinates of point B. Point B is the intersection of Y = 3 
and X -f- 2Y = 8. Hence, B is given by (2, 3) and, in turn, Z max = 2(2) 
+ 5(3) = 19. 

Geometric Interpretation of the Simplex Method. In order to 
exhibit, geometrically, what happens when one solves the problem by means 
of the simplex technique, the simplex solution of the example of Fig. 8 is 
given in Tables 27a-c. We see from those tables that the solution progresses 
from the point (X = X x = 0, Y = X 2 = 0) to the point (X = X x = 0, F 
s X 2 = 3) to the point (X = X x = 2, Y = X 2 = 3); i.e., referring to 
Fig. 8, from point (origin) to point A to point B. 

Mathematically, polygon OABCD (Fig. 6) constitutes a convex set of 
points; i.e., given any two points in the polygon, the line segment joining 
them is also in the polygon. An extreme point of a convex set is any point 
in the convex set which does not lie on a line segment joining some two other 
points of the set. Thus, the extreme points of polygon OABCD are points 
0, A, B, C, and D. The optimum solution to the linear programming 
problem will be at an extreme point and this optimum (extreme) point is 
reached by proceeding from one extreme point to another. Note that, in 
the example discussed here, the solution proceeded from extreme point 
(Table 27a) to extreme point A (Table 276) and, finally, to extreme point 
B (Table 27c). 

More Than One Optimum Solution. If the example is now changed 
slightly to read: 

Problem 2. To determine X, Y 3: which maximize Z = X + 2F 
subject to the restrictions 

F^3, 

X + 2F ^ 8, 

then Fig. 9 shows that the solution is given by either extreme point B or 
extreme point C. This is because X -\- 2Y = 8 is both a boundary line of 



Table 27. Simplex Method 
(a) Feasible Solution Corresponding to X = 0, Y = in Fig. 8 



\c f 
















2 


5 


! 


Basis 


Po 


p 3 


p 4 


p 5 


P{ 


P 2 





P S 


4 


1 








1 








Pa 


3 





1 








1 





P 5 


8 








1 


.1 


2 




Zj 






















Zj — Cj 














-2 


-5 



(6) Feasible Solution Corresponding to X = 0, 1 


7 = Sin 


Fig. 8 


















2 


5 




Basis 


Po 


P S 


P 4 


Pb 


Pi 


P 2 





Pz 


4 


1 








1 





5 


P 2 


3 





1 








1 





P 5 


2 





-2 


1 


1 







Zj — Cj 


15 





5 





-2 






(c) Maximum Feasible Solution Corresponding to X = 2,Y = 3 in Fig. 8 



\Cj 
















2 


5 




Basis 


Po 


P 3 


P 4 


P 5 


Pi 


P 2 





P 3 


2 


1 


2 


-1 








5 


P 2 


3 





1 








1 


2 


Pi 


2 





-2 


1 


1 







Zij — L>j 


19 





1 


2 
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the polygon OABCD and also a member of the family of parallel lines Z 
= X + 2F. Hence B = (2, 3) and C = (4, 2) both constitute solutions 
and yield the answer Z max = 8. 





f 


X=4 
Y=3 

c 

^ ^ X + 2Y=8 


A 


^ 

\ 

^ 





X+2Y=0 


^ X+2Y=6 
X+2Y=4 



Fig. 9. Geometric solution of linear programming problem with more than one optimum 

solution (Ref. 1). 

Furthermore, any convex linear combination of B and C will also be a 
solution, namely, any point on the line segment BC. 

The Dual Problem of Linear Programming 

By a dual theorem of linear programming, one has a choice of two prob- 
lems to solve instead of just one. This is because every linear programming 
problem has a dual problem such that one involves maximizing a linear 
function and the other involves minimizing a linear function. Further- 
more, if one solves a linear programming problem by the simplex technique, 
a tableau corresponding to an optimum solution automatically contains a 
solution to the dual problem. Thus, one is free to work with either the 
stated problem or its dual. 

The dual problem of linear programming is illustrated by the example 
given earlier (eq. 74), namely: 

Problem. Determine X, Y ^ which maximize Z = 2X + 5F subject 
to 

X g4, 



(74) 



7^3, 

X + 27 £8. 



This problem may be displayed in tabular form as is done in Table 28, that 
is, the restrictions may be read off by interpreting a light vertical line as -f- 
and the heavy vertical line as ^. Furthermore, the function to be max- 
imized is given by the bottom row, namely 2X + 5 Y. To obtain the dual 
problem, extend Table 28 as is done in Table 29. Then, by reading down 
each column as indicated, obtain the dual problem, namely : .. 



15-68 



OPERATIONS RESEARCH 



Table 28. Tabular Form of Problem 



Max 



X 


Y 




1 





4 





1 


3 


1 


2 


8 


2 


5 





Table 29. Dual Problem in Tabular Form 

< 





X 


Y 


Min 


Wi 


1 





4 


w 2 





1 


3 


w s 


1 


2 


8 


Max 


2 


5 





Dual Problem (see Table 29). Minimize g = 4Wi + SW 2 + 8W 3 

subject to 

TTi + W 3 ^ 2, 
(75) 

W 2 + 2W 3 ^ 5. 

The inequalities, ^, are converted to equalities by the subtraction of 
non-negative slack variables. Then, since —1 cannot be entered into a 
basis, one may also add artificial variables to provide for the basis. Thus, 
Wi + W 3 ^ 2 is first converted into W t + W 3 - TF 4 = 2. Then the arti- 
ficial variable W§ may be added to provide W\ + W 3 — W 4 + W 6 = 2. 
For a detailed discussion, see Charnes, Cooper, and Henderson, Ref. 36. 
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If one returns to the simplex solution of the maximization problem of 
Table 27c, one sees that the following results are given : 



and 





^max 


= 19, 


X 1 = 2, 




Zi - Ci = 0, 


X 2 = 3, 




Z 2 — ^2 = 0, 


X 3 = 2, 




z 3 - c 3 = o, 


X 4 = 0, 




Z 4 — G 4 = 1, 


x 5 = o, 




Z5 — C5 = 2. 



(76) 



Now, X 3 , X4, and X 5 correspond to slack variables. Hence, if one starts 
with the first slack variable and renumbers the Zj — Cj in order, and de- 
notes these reordered Zj — Cj by Z'j — Cj, one obtains 

Z'i — C'i = (corresponding to former Z 3 — C 3 ), 

Z'2 — C f 2 = 1 (corresponding to former Z 4 — C4), 

(77) Z' 3 — C' 3 = 2 (corresponding to former Z 5 — C 5 ), 

Z' 4 — C 4 = (corresponding to former Z± — Ci), 

Z' 5 — C" 5 = (corresponding to former Z 2 — C 2 ). 

Setting Wj = Z'j — Cj gives the solution to the dual minimization prob- 
lem ; that is, if the minimization problem were to be solved by the simplex 
technique, the following results would be obtained : 



and 



(78) 



where the bj are the corresponding coefficients of the Wj in the minimization 
function. 

Conversely, given the solution to the minimization problem (i.e., given 
eq. 78), the solution to the dual maximization problem can be determined • 
by starting with the first slack variable W4 and relabeling the — (gj — bj) 
in order. Hence, solution eqs. (76) would result. 





<7min 


= 19, 


w, = 0, 




-(0i-6i) = 2, 


w 2 = 1, 




-(g 2 -b 2 ) = 0, 


W 3 = 2, 




-(g 3 -h) = 0, 


w 4 = o, 




-(94-h) = 2, 


TF 5 = 0, 




- (95 - h) = 3, 
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For dual problems,, it can be shown that Z max = g m in', in other words, 
that the two problems are equivalent (see Ref. 36 and Ref. 40, Chap. XIX). 
Hence, in solving a linear programming problem, one is free to work with 
either the stated problem or its dual. Since, as a rule of thumb, the number 
of iterations required to solve a linear programming problem is equal to one to 
one and a half times the number of rows (i.e., restrictions), one can, by an 
appropriate choice, facilitate the computation somewhat, especially in 
such cases where there exists a sizable difference in the number of rows for 
each of the two problems. 

A Short Cut in Solving Linear Programming Problems 

One of the many advantages of both the transportation and simplex 
techniques is that judgment can be used to good advantage in facilitating 
the computations required in order to arrive at an optimum solution. In 
the transportation problem involving m rows and n columns, the use of 
judgment (or a good guess) simply requires designating m + n — 1 cells 
which are expected to correspond to a solution. After these m + n — 1 
cells have been selected, proceed as in the transportation technique, first 
filling in these cells with circled numbers and then "evaluating" the re- 
maining cells to determine whether or not the solution is an optimum one. 

Consider the problem of Fig. 6 and eq. (74). It will be shown that given 
a "good" guess, the corresponding simplex matrix can be constructed. 
Then one proceeds to the optimum solution, if the solution guessed is not 
already optimum. This demonstrates how one may utilize judgment in 
the general linear programming problem (using the simplex technique). 

Problem. To determine X, Y ^ which maximize Z = 2X -f- 5Y 
subject to 

X^4, 

(79) 7^3, 

I + 27g8. 

Converting this system of inequalities to equalities by means of slack var- 
iables $3, S±, and $ 5 yields 

X + S 3 = 4, 

(80) Y 4- S 4 = 3, 

X + 2F + £ 5 = 8. 

Now, suppose that one "guesses" or has reason to believe that the optimum 
solution is such that it will not involve X; i.e., that the final solution will 
consist of Y, S 3 , and S 5 . This means, accordingly, that X = and £ 4 = 0. 
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Hence, to obtain the "solution," i.e., the elements of the basis that would 
appear in the P column of the simplex tableau, one needs only to set X = 
and Si = in eqs. (80), yielding 

Ss =4, 

(81) Y = 3, 

2Y + S 5 = 8, 
so that 

(82) Y = 3, £ 3 = 4, S 5 = 2. 

These values are then entered in the simplex tableau (see Table 30) under 
the column labeled P . Note that P 2 corresponds to Y. 

Table 30. Feasible Solution, Short-Cut Approach 















2 


5 




Basis 


Po 


P3 


p 4 


P5 


Pi 


P 2 





P3 


4 


1 








1 





5 


PtiT) 


3 





1 








1 





Pb 


2 





-2 


1 


1 







Zs 


15 





5 








5 




Zj ~~ Cy 


15 





5 





-2 






Next, construct the body of the simplex matrix. Since each value of 
Zj — Cj corresponds to the minimum cost of deviating from the optimum 
program by one unit of Xj, one can determine, for each j, the corresponding 
Zj — Cj and the X^ which appear in that column. For example, consider 
that one will deviate from the program of Y = 3, $3 = 4, and *S 5 = 2 by- 
insisting that X = 1. One then needs to determine the changes in Y, *S 3 , 
and S 5 which result from the unit change in X. Therefore, solve 



1 + S 3 = 4, 



(83) 



7 = 3, 

1 + 2F + S 5 = 8, 
which result from eqs. (80) by letting X = 1 and £ 4 = 0. 
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Solving eqs. (83) yields 

(84) X = 1, Y = 3, S 3 = 3, S 5 = 1. 

Comparing eqs. (82) with (84) then shows that the following changes in 
Y, S 3 , and S 5 occur because of a unit change in X: 

(85) A7 = 0, AS 3 = 1, - AS 5 = 1. 

Therefore, in setting up a simplex tableau (see Table 30), these values 
would be inserted under the column labeled P x which corresponds to the 
variable X. 



Similarly, 


for # 4 solve 


#3 = 4, 


(86) 




7+1 = 3, 


This yields 




2F + S 5 = 8. 


(87) 


F = 2, 


S 3 =4, 


so that 






(88) 


AY = 1, 


AS 3 = 0, 



£5 = 4, 

AS 5 = -2. 

Insert these values in column P4 of Table 30. 

Next, since P 2 , P3, and P 5 are in the basis, complete the corresponding 
columns (as is done in Table 30) by inserting 0's and l's in the appropriate 
places. 

Finally, compute the Zj — C/s to determine whether the "solution" is 
optimum. This is done as at the outset of any simplex solution; i.e., first 
compute Zj by 

(89) Zj = 2 dXij 

i 

and then subtract the corresponding Cj. Since P 2 , P3, and P 5 are in the 
basis, Z 2 — C 2 , Z s — C s , and Z 5 — C 5 are all equal to zero. Additionally, 
applying eq. (89), yields 

Z 1 -C 1 = 1(0) + 0(5) + 1(0) - 2 = -2, 

Z 4 - C 4 = 0(0) + 1(5) + (-2)(0) -0 = 5. 

Thus Table 30 is completed and, not having an optimum solution (owing 
to Z\ — C\ being negative), one can proceed to obtain the optimum solu- 
tion as before. 
The reader should note that Table 30 is identical with Table 276 and was 
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generated without a tableau such as is given in Table 27a. The same tech- 
nique can also be applied to larger size problems so that, with a good esti- 
mate of the variables which will make up the solution, a great amount of 
computation might be eliminated. 

The Assignment Problem (See Ref. 1, Chap. 12) 

The assignment problem is a special linear programming problem which 
may be stated mathematically as follows: 
Determine Xij which minimize 

i'j 

subject to 

Xij = X{j , z, j = 1, 2, • • •, n 

n n 

V Xn = V Xn =1, i = 1, • • •, n; j = 1, • • •, n. 
i=l i=i 

In other words, the assignment problem is such that: 

(a) x^ = 1, if the ith facility is assigned to the jth. job; 0, otherwise. 

(b) Each row and column of the solution matrix will have one element 
unity and all other elements zero. 

For both the assignment problem and the transportation problem, so- 
called "methods of reduced matrices" exist which enable one to obtain the 
optimum solution with great ease. 

5. WAITING TIME MODELS 

Problem Statement. A waiting time problem arises when either units 
requiring service or the facilities which are available for providing service 
stand idle, i.e., wait. Problems involving waiting time fall into two differ- 
ent types, depending on their structure. 

a. Waiting line problems involve arrivals which are randomly spaced 
and/or service time of random duration. This class of problems includes 
situations requiring either determination of the optimum number of service 
facilities or the optimum arrival rate (or times of arrival), or both. The 
solution of these "facility and scheduling" problems is obtained through 
what is called waiting line theory or (from the British) queuing theory. 

Queuing theory dates back to the work of A. K. Erlang, who in 1908 
published Use of Waiting-Line Theory in the Danish Telephone System. In 
Erlang's and subsequent work up to approximately 1945, applications were 
restricted in the main to the operation of telephone systems. Since then 
the theory has been extended and applied to a wide variety of phenomena. 
See Ref. 42 and Ref. 1, Chap. 14. 
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Reference 42 also contains an excellent list of activities to which queuing 
theory has been applied, a description of the use of the Monte Carlo tech- 
nique in solving queuing problems, and a comprehensive list of references. 

b. Sequencing. The second type of waiting time problem is not concerned 
with either controlling the times of arrivals or the number of facilities, but 
rather is concerned with the order or sequence in which service is provided 
to available units by a series of service points. This is the so-called sequenc- 
ing problem. See Ref. 1, Chap. 16. 

For a discussion of related problems such as the (assembly) line-balancing 
problem and the traveling salesman (or routing) problem, see Ref. 1. 

Problem Characteristics of Queuing Models 

Every queuing or waiting line problem can be characterized by the fol- 
lowing factors: 

1. Input, the manner in which units arrive and become part of the wait- 
ing line. 

2. Stations, the number of service units (or channels) operating on the 
units requiring service. 

3. Service policy, limitations on the amount of service that can be ren- 
dered or is allowed. 

4. Queue discipline, the order in which units are served, e.g., first come, 
first served; random selection for service; priority. 

5. Output, the service provided and its duration. To specify a queue 
completely, all five factors must be described. 

Notation (see Ref. 42). 

X mean arrival rate (number of arrivals per unit time) 

H mean service rate per channel 

c the number of service channels 

Cf mean number of free service channels 

n number of units (customers) in the system 

k number of phases in the Erlang service case 

p utilization factor for service facility: p = X/cp. 

P n (t) the probability that there be, at time t, exactly n units in the system, both 

waiting and in service 
p n the steady-state (time-independent) probability that there be n units in the 

system, both waiting and in service: 

n=oo n=co 

E Pn(t) = £ Pn = 1 
n=0 «=0 

cp traffic intensity in erlangs: 

\ n—c — 1 w=oo 

c P = - = c - Cf = £ np n + £ cp„ 

A* n=0 n=c 

P( = 0) the probability of no waiting 
P(>0) the probability of any waiting 
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P(>t) the probability of waiting greater than time r 

L the average number of units in the system, both waiting and in service: 

n=co 

L = Y, n Pn 
n=0 

L q the average number of units waiting in the queue: 

00 

Lq = 2 ( n — c )Vn = L — C +Cf 

n=c 

W the average waiting time in the system: 

W = - f rdP{>r) 

A(t) cumulative distribution of times between arrivals with density function a(t) 

B(t) cumulative distribution of service or holding times with density function b(t) 

b/c(t) probability density for kth Erlang distribution 

Input. Arrivals or inputs into a queuing system may occur at intervals 
of regular length. For such cases the cumulative distribution of time inter- 
vals between arrivals is given by the uniform distribution 

A(t) = Oforf < t ; 1 for t ^ t . 

If the input distribution is of Poisson type, the time intervals between 
arrivals are exponentially distributed. The cumulative distribution is then 
given by 

A(t) = 1 - e~ u . 

An intermediate type of input may be described by the Erlangian fre- 
quency distribution of times between arrivals 



b*(0 = 



r(fc)J 



This yields the exponential distribution when k = 1 and the uniform dis- 
tribution when k becomes infinitely large. 

As Saaty points out (Ref. 42), the normal distribution also produces a 
good fit to arrival data in some practical problems. 

Output (Service or Holding Times). Distributions of service or 
holding times are defined as for arrivals or inputs. In practice, Poisson in- 
puts and exponential service times occur very frequently. 

Assumptions Leading to a Poisson Input. (See Ref. 42.) One has 
a Poisson input when the following assumptions are satisfied: 

1. The total number of arrivals during any given time interval is inde- 
pendent of the number of arrivals that have already occurred prior to the 
beginning of the interval. 
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2. For any interval (t, t + dt), the probability that exactly one arrival 
will occur is X dt + 0(dt 2 ), where X is a constant, while the probability that 
more than one arrival will occur is of the order of dt 2 and may be neglected. 

For a further discussion of the Poisson input and properties of a Poisson 
process, see Refs. 42 and 43. 

Assumptions Leading to an Exponential Holding Time Distribu- 
tion. If a channel is occupied at time t, the probability that it will become 
free during the following time interval dt is given by /x dt, where ju is a 
constant. (See Ref. 42.) 

It follows that the frequency function of the service times is ne —flt , while 
the mean duration of service is l//z, since the expected value of t is 

E(t) = n I te-^dt = -• 
Queuing Models 

To date, there have been essentially two different theoretical approaches 
to queuing, one through differential difference equations due to Erlang and 
the other through integral equations as studied by Lindley. The first ap- 
proach may be illustrated by means of a single channel queuing system 
with both X and n constant. A Poisson input, exponential holding time, 
first-come, first-served single channel queue is assumed. 

Differential Difference Equations. If the operation starts with no 
items in the queue, then the following equations describe the given system. 
(See Ref. 1.) 

P (t + dt) = P (t)(l -\dl)+ Pi(0m dt (n = 0), 

P n (t + dt) = P n (t)[l - (X + fi) dt) + P n _x(0X dt + P n+1 (t)n dt (n ^ 1). 

By transposing and passing to the limit with respect to dt, these equations 
become 

dP (t) 

= -XP o (0 + mPi(0 (n = 0), 



dt 

dP n (t) 

dt 



= -(X + n)P n (t) + XP„_!(0 + nP n+ i(t) (n ^ 1). 



The time-independent steady-state solution is obtained either by solving 
these time-dependent transient equations and letting t — > °o in the solu- 
tion, or by setting the derivatives with respect to time equal to zero, and 
solving the resulting steady-state equations. The latter approach yields, 
successively : 

Xpo = m, 

(X -f" n)p n = Xpn-l + MPn-1- 
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By mathematical induction, these formulas then reduce to the single 
equation : 

Vn = P w (l - p), 

where p = X/p, since c = 1. 

The expected number of units in the system is given by 

00 00 

L = X) n Vn = (1 - p) Z) np n = p/(l - p). 

M=0 tt=0 

The expected number of units in the line is given by 

L q = L- P = p 2 /(l - p). 
The expected waiting time is given by (see Ref . 42) 



W = f rdP{<r) = — 

J p(l — 



P) 



where P(<t), the probability that an arrival waits a time less than r, is 
given by 

P(<r) = 1 - pe-" T(1 - p) . 

Integral Equations. For a resume and illustration of the development 
and application of integral equations to queuing problems, see Ref. 42. 

Queuing Theory Formulas. Single Channel 

1. Poisson Input, First-Come, First-Served, Arbitrary Holding 
Time. The expected total number of units waiting (queue plus service) 
is given by 

Variance (t) + (1/ M ) 2 

L = A + u H » 

(2/XX1/X - 1/ M ) 

where £ has the holding time distribution. 
This may also be expressed as 

L = \[W + (1/p,)], 
where 

W = [1 4- (s M ) 2 ], 

2 M (l-p) L VWJ ' 

where s is the standard deviation of the service time. 
(a) Exponential Holding Time Distribution. Here 

Vn = (X/p.) n (l - X//0 
and 

L = p/(1 - p). 
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The expected number waiting in line is given by 

L q = p 2 (l - p). 
P(>r) = P e^ p - l)T 
X P 



Furthermore, 
and 



W = 



n(n — X) m(1 — p) 
The expected number waiting, of those delayed, is 

1 



(1 - p) 
The expected waiting time of those delayed is 

W 1 



P(>0) M (l - p) 
See Refs. 42-44. 

(b) Constant Holding Time Distribution (Refs. 42 and 45). The steady- 
state equations are : 



Vo = 1 - P, 

Pl = (1 - p)(ef - 1), 

n 

p» = a -p) E (-i) w -V" 

Here, 



(M 



n — A; 



+ 



(kp) 



n — k — 1 -i 



L(n-fc)! (n-fc-l)U 



(n ^ 2). 



^(>r) = p E ^^^[-pCmt - i)]'/*!, 

where k is the largest integer less than or equal to /xr. 

P 



TF = X 



^ 



(-;) 



2p(1 - p) 

Finally, the expected waiting time of those delayed is 

1 



2m(1 - p) 

(c) Poisson Input, Erlangian Holding Time Distribution. The probability 
density function for the fcth Erlang holding time distribution is given by 



b k (t) = 



L T(k) 



p — \xkttk — 1 
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The steady-state equations are (Ref. 45) 

Xpo = nPi (n = 0), 

(X -f- y)Vn = MPn+l + Xpn-fc (n ^ 1). 

p( P + 2k - pk) 



Here 



L = 
W = 



2fc(l - p) 
P (k + 1) 



2/ifc(l - p) 

2. Priority Discipline: Arbitrary Holding Time, Nonpreemptive 
Service (Refs. 42 and 46). 

(a) Finite Number of Priorities, N. Assume a system with Poisson input 
for the /cth priority with arrival rate \k, arbitrary holding time with service 
rate p.k, and a priority queue discipline. Items of different types enter the 
system with assigned priorities for service. Whenever the system is free 
to service an item, it selects items of highest priority on a first-come, first- 
served basis. However, if an item of higher priority enters the system while 
one of lower priority is in service, this service is not preempted, i.e., sent 
back to the waiting line. For this situation, 

W 



W k = 



where 



(1 — OVb-lXl — (Tfc) 



Pi = —' 

Mi 

N 

x = E K 

i=l 
k 

<Tk = Z2 Pi < 1) 

i=l 

W = JX f t 2 dF(t), 
Jo 

1 N 
m =-E X«0, 

X t=i 

and where Fk(t) is the cumulative holding time distribution function for 
the /cth priority. 

The expected length of the line is given by 

N 

L = £ XiWi. 
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(b) Two Priorities, Preemptive Service, Exponential Holding Time. (See 
Refs. 42 and 47.) Priority 1 and 2 calls arrive at a single channel with 
arrival rates Xi and X 2 , respectively. Both priorities have Poisson arrival 
distribution. Priority 1 calls in the queue enter the channel before all 
priority 2 calls in queue and replace any priority 2 calls in the channel on 
their arrival. The priority 2 call in the channel then reenters the queue. 

Priority 1 and 2 calls have exponential service time distribution with 
service rates m and /z 2 respectively. Let pi = Xi/Vi, and p 2 = X 2 /ju 2 where 
Xi/mi + X 2 //x 2 < 1. 

Let p n m be the probability that n priority 1 calls and to priority 2 calls are 
in the queue. The steady-state equations are: 

MlPn+1™ - 0"1 + Xj + \ 2 ) Vn m + \ip n -i m + ^PrT^ = (TO, U > 0), 

MiPi m + n 2 Po m+1 - (m 2 + Xi + X 2 )p m + X 2 p m_1 =0 (n = 0, m > 0), 
mVn+i° - (mi + Xi + X 2 )p n ° 4- X^^ = (m = 0, n > 0), 
mPi° + /W - (Xi + X 2 )po° = (m = n = 0). 
The expected number waiting, first priority, is given by 

Pi 
1-Pi' 
The expected number waiting, second priority, is given by 

1 — Pi + (mi/m2)pi 



P2 



(1 — Pi)(l - Pi — P2) 



(c) Continuous Number of Priorities (Ref. 42). For an excellent discus- 
sion of results for a single channel priority queuing system with application 
to machine breakdowns, see Ref. 48. The number of available machines 
is assumed to be infinite. Priorities are assigned according to the length of 
time needed to service a machine with higher priorities being assigned to 
shorter jobs. Since the length of service time may correspond to any real 
number, a continuous number of priorities exists. 

3. Random Selection for Service: Impatient Customers, Ex- 
ponential Holding Time. (See Refs. 42 and 49.) 

Assumptions. Poisson input, exponential holding time, random selection 
for service, items leave after a wait of time T . 

Steady-State Equations. 

-\p + ( M + Ci) Pl = (n = 0), 
Pn-i - (X + ix + C n )p n + (m + C n+1 )p n+1 =0 (n ^ 0), 
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where C n is the average rate at which customers leave when there are n 
customers in the system and where po is obtained from 

Z) Vn = 1. 

n = 

Solution. 

n 

Vn = X w p II 0* + Ck)- 1 (n = 1, 2, • • •), 

_ Atexp (—iiTo/n) 
1 - exp (—pTo/ri) 

4. Limited Source: Exponential Holding Time. (See Refs. 42 
and 43.) 

Assumptions. Input from a source having only a finite number m of cus- 
tomers. Exponential service time, single channel (servicing of m machines). 
Steady-State Equations. 

m\po = npi (n = 0) 

[(to - n)X + tAVn = (to - n + l)Xp n _i + Wn+i (1 ^ n ^ to - 1), 

MPm = Xpm-i (n ^ to). 
Solution. 

p n = m(m — l){m — n+ l)(\/n) n p , 

m 
Vo = 1 - Z) Pn, 

L = to - [(X + /OAltt - Po). 

5. Constant Input: Exponential Holding Time. (See Refs. 42 
and 50.) For constant input at intervals of length 5, 

Pn = Po(l - Po) n , 
where p is given by 

1 - po = exp (-np 8). 
Furthermore, 

P(>t) = (1 - po) exp (-HP08), 
and 



TF 



= - f rdP{>r) = (1 -p )/nPo- 
Jo 



6. Queue Length-Dependent Parameters and Time-Dependent 
Parameters. For an excellent resume of queuing results for queue length- 
dependent and time-dependent parameters, see Ref. 42. See also Refs. 
51-53. 



Channel 1 


Channel 2 


Total System 


Xi 


X 2 


Xl 2 x 2 2 


1 — Xi 


1 — x 2 


1 — Xi 1 — x 2 


Xi 


X 2 


X\ + x 2 


Xi 


x 2 


Xl x 2 
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Queuing Theory Formulas. Two Channels in Series (See Ref. 54.) 

Exponential Holding Times. (See Ref. 42.) 

Assumptions. Poisson input with mean X, two channels in series with 
exponential holding times, m and /j, 2 , respectively. After finishing service 
at the first gate, the customer moves on to the second gate. 

(a) Unlimited Input. The average distribution of customers throughout 
the system is given in the following table. 



Average number of customers 
waiting for service 

Average number of customers 
being served 

Average total number of 

customers 1 - Xl 1 - x 2 1 - x x ^ 1 - x 2 

The steady-state solution giving the probability that there are ni cus- 
tomers waiting at the first gate and n 2 at the second is given by 

p(ni, n 2 ) = {x l Y^x 2 ) n Kl ~ *i)(l ~ x 2 ), 
where 

xi = X/yui < 1 and x 2 = \/n 2 < 1. 

The probabilities of having n customers waiting at the first channel and 
at the second channel are, respectively, 

Pi(n) = rci n (l - xi), 

p 2 (n) = x 2 n (l - x 2 ). 

(b) Limited Input. For a resume of results for limited input, see Ref. 42. 

Queuing Theory Formulas. Three Channels in Series 

Results for the case of three channels in series can be found in Ref. 54. 

Queuing Theory Formulas. Multiple Channels in Parallel, Poisson Input 

An excellent resume of results for both a finite and infinite number of 
channels can be found in Ref. 42. For the case with a finite number of 
channels, this includes: (a) identical exponential holding times, (b) identi- 
cal constant holding times, (c) (priority discipline) different Poisson inputs, 
a finite number of priorities with the same exponential holding time (non- 
preemptive), and (d) (limited source) exponential holding time. 
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The resume" for an infinite number of channels covers: (a) exponential 
holding time and (b) limited source, exponential holding time. 

Sequencing Models 

For a detailed discussion of sequencing models, see Ref. 1, Chap. 16. 
Only a few results are presented here. 

1. Two-Station and n Jobs, No Passing. Consider the case of 
n jobs to be processed on two machines, A and B, with each job requiring 
the same sequence of operations and no passing allowed. The order (se- 
quence) in which jobs are processed on machine A must be retained in 
processing these same jobs on machine B. It is assumed that material can 
be held between work stations so that, in the meantime, the preceding work 
station is left clear to start work on another job. It is further assumed 
(without loss of generality) that all jobs must first go to machine A and 
then machine B. 

Let Ai — time required by job i on machine A, 
B{ = time required by job i on machine B, 
T = total elapsed time for jobs 1,2, • • • , n, 
X{ = idle time on machine B from end of job i — 1 to start of job i. 
The sequencing problem is to minimize T, the total elapsed time. 

The total elapsed time may be expressed as 

n n 

T=T,B i +J2 X im 

For any given set of items, 2™ = i.B; is constant; therefore, the problem of 
minimizing T is equivalent to that of minimizing 

n 

Dn(S) = £ X i} 
i=l 

where D n (S) is a function of the sequence S. 

Procedure for Finding the Optimum Sequence. A procedure for 
finding the optimum sequence for two stations, n jobs, and no passing is 
due to Johnson (Ref. 55) and can be described by means of the example 
represented in Table 31. 

Table 31. Machine Times (in Hours) for Five Jobs and Two Machines 

i Ai Bi 

1 3 6 

2 7 2 

3 4 7 

4 5 3 

5 7 4 
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Step 1. Examine the A/s and Bi's and find the smallest value [min 
(A{, Bi)]. In this illustrative case, this value is B 2 = 2. 

Step 2. If the value determined falls in the A{ column, schedule this job 
first on machine A. If the value falls in the Bi column (as it does in this 
case), schedule the job last on machine A. Hence, job 2 goes last on 
machine A. 

Step 8. Cross off the job just assigned and continue by repeating the 
procedure given in steps 1 and 2. In case of a tie, choose any job among 
those tied. In this illustrative case, once job 2 is assigned, the minimum 
value which remains is 3, and it occurs in A\ and J3 4 . There is a choice, so 
arbitrarily select A\. Then job 1 goes on machine A first. Now B 4 is the 
minimum remaining value. Hence, job 4 goes on machine A next to last. 
The minimum remaining value is 4, and it occurs in A 3 and B 5 . Then 
job 3 can be put on machine A second and job 5 on third to the last. The 
resulting sequence is optimum and is 1, 3, 5, 4, 2. 

2. Three Stations and n Jobs, No Passing. 

Let Yi = the idle time on the third machine before it starts work on the 
ith job, 
C{ = working time of the third machine on the ith. job. 
The total elapsed time for three stations and n jobs (no passing) is given by 

n n 

T = £ Ct + E Yt. 

i=i i=\ 

Since 2? =1 C; is fixed, the problem is to minimize S? = iFj. Johnson 
(Ref. 55) has found an optimum solution to this problem for the special 
case where either (1) min A t - ^ max Bi or (2) min C{ ^ max B{. The first 
of these conditions is satisfied by means of an exact equality in the illustra- 
tive data given in Table 32. 

Table 32. Machine Times (in Hours) for Five Jobs and Three Machines 

1 Ai t>i Ov 



1 


8 


5 


4 


2 


10 


6 


9 


3 


6 


2 


8 


4 


7 


3 


6 


5 


11 


4 


5 



To obtain an optimal sequence a new table, such as Table 33, is formed. 
Then the procedure (described in the preceding) for obtaining an optimum 
sequence for two stations is applied to Table 33. In this case, the following 
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Table 33. Sums of Machine Times (in Hours) for Five Jobs for First and 
Intermediate Machines and for Intermediate and Last Machines 



i 


At + B t 


Bi + 


1 


13 


9 


2 


16 


15 


3 


8 


10 


4 


10 


9 


5 


15 


9 



sequences arise and are optimum for the originally cited three-station 
problem : 

3, 2, 1, 4, 5 3, 2, 1, 5, 4 

3, 2, 4, 5, 1 3, 2, 5, 1, 4 

3, 2, 4, 1, 5 3, 2, 5, 4, 1 

In situations where the conditions min Ai 2s max B{ or min C; ^ max B{ 
do not hold, no general procedure is available as yet for obtaining an opti- 
mum sequence. It follows that no general solution is yet available for the 
more general problem of n jobs and m machines, each job following an 
identical route with no passing allowed. However, the following statement 
holds: For optimum sequences (the criterion being the total elapsed time), 
the total idle time of the last machine must be minimized. 

3. Identical Routing, Passing Permitted. Although each of n jobs 
may have to pass through each of m stations according to a specific route, 
the process characteristics do not always require that the order in which n 
jobs pass through each of the stations be identical, i.e., passing is permitted. 
Bellman (Ref . 56) and Johnson (Ref . 55) have shown, that for two or three 
station processes, the optimum sequence always involves the same ordering 
of jobs over each station. This result does not necessarily hold where 
more than three stations are involved. 

4. Different Routing. In many production operations, particularly in 
job shops, the various jobs which must be done require different routing 
through the work stations or centers. 

The problem of determining the optimum sequence for two jobs which 
have to be processed on m machines using two different routes has been 
treated by Akers and Friedman (Ref. 57) who, by means of Boolean alge- 
bra, have developed a technique for eliminating sequences which are tech- 
nologically unfeasible. Their technique yields a subset of sequences, one 
or more of which is optimum. 
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The Akers-Friedman technique can also be extended to apply to the case 
of n jobs and m stations. See Refs. 56 and 57. 

6. REPLACEMENT MODELS 

Problem Statement. The theory of replacement is concerned with 
the prediction of replacement costs and the determination of the most eco- 
nomic replacement policy. There are two basic types of replacement prob- 
lems concerned with (a) items that deteriorate with age and/or use and, (b) 
items with probabilistic life spans and with efficiencies that do not decline 
over their life spans. 

For type (a) items that deteriorate or degenerate with age and use, the 
problem is to determine when to replace equipment so as to minimize the sum 
of costs due to loss of efficiency, on the one hand, and cost of new equip- 
ment, on the other hand. 

For items whose efficiency declines over their life spans (e.g., machine 
tools, vehicles), prediction of costs involves determining those factors which 
contribute to increased operating cost, forced idle time, increased scrap, 
increased repair, etc. 

The alternative to the increased cost of operating aging equipment is the 
cost of replacing old equipment with new. There is some age at which re- 
placement of old equipment is more economical than continuation at the 
increased operating cost. At that age, the saving from the use of new equip- 
ment more than compensates for its initial cost. 

For type (b) items that do not essentially deteriorate with age and use, 
but which have probabilistic life spans (e.g., light bulbs or radio tubes), the 
problem is to determine when and how to replace the items (i.e., individually 
or in groups) so as to minimize the sum of costs of (1) the items, (2) replac- 
ing items after failure, and (3) group replacements. 

For a group of items with a probabilistic life span, the prediction of costs 
involves the estimation of the probability distribution of life spans and cal- 
culation from these of the predicted number of failures as a function of the 
age of the group of items. For several schemes for approximating the 
number of failures, see Refs. 43 and 58-62. 

For a complete discussion of both types of replacement problems, see 
Ref. 1, Part VII. 

Replacement of Items That Deteriorate 

The measure of efficiency used as a basis for determining optimum replace- 
ment decision rules is the discounted value of all future costs associated with 
any replacement policy. Discounted cost is the amount required at the 
time of the policy decision to build up a fund at compound interest large 
enough to pay the pertinent cost when due. 
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In general, the costs included in the replacement decisions cited here are 
all costs that depend upon the choice or age of the machine. See Ref. 1, 
Chap. 17, for a discussion of the relevant costs in replacement theory con- 
siderations. 

Cost Equation. Consider a series of time periods 1, 2, 3, 4, • • •, of 
equal length, and let the costs incurred in these periods be C x , C 2 , C 3 , C 4 , 
• • • , respectively. It is assumed throughout that, relevant to items that dete- 
riorate, these costs are monotonically increasing with time. Assume that each 
cost is paid at the beginning of the period in which it is incurred, that the 
initial cost of new equipment is A, and that the cost of investment is 100r% 
per period. 

The discounted value K n of all future costs associated with a policy of 
replacing equipment after each n periods is given by 



(90) K n = 



A + d + 



Ci 



+ 



1 + r 
A + Ci 



+ 



+ 



C, 



(1 + rf 
C 2 



+ •■■ + 



l^n 



L(l + r) n (1 + r) n+1 
Equation (90) may also be written as 



+ •■• + 



(1 + r) n ~ l . 



(1 + 



271—1 



+ 



(91) 
or, if 
(92) 
then 

(93) 



Kn = 



A + £ [CV(1 + ry- 1 ] 
i=i 

1 - [1/(1 + r)]» 

1 
X = 



1 + r 



K n = 



A + E CiX'- 1 

i=l 

1 - x n 



Now, if the best policy is replacement every n time periods, the two 
inequalities 



(94) 



K n+1 - K n > and K n - X - K n > 



must hold. Furthermore, for the case where the C n are monotonic increas- 
ing, these conditions are sufficient as well as necessary ones for K n to be 
minimum. 
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From eq. (93), K n -i — K n > is equivalent to (see Ref. 1) 

(95) C n < (1 - X)K n , 
and K n+ i — K n > is equivalent to 

(96) C n+1 >(1 - X)K n . 

These inequalities, (95) and (96), may also be written as: 
(A + CQ + CaX+.-.+ Cn-iX"" 2 

( 7) Cn< i + x + x 2 +.-.+ x»- 2 

and 

(98) c„ +1 > (A+c ' )+c f + - +c -r . 

where the right-hand terms are the weighted averages of all costs up to and 
including the (n — l)st and the nth periods, respectively. 

Decision Rules. As a result of these two inequalities, the following 
decision rules for minimizing costs may be stated : 

1. Do not replace if the next period's cost is less than the weighted average 
of previous costs. 

2. Replace if the next period's cost is greater than the weighted average 
of previous costs. 

For further discussion and a geometric interpretation of these decision 
rules, and also an illustration of their use, see Ref. 1, Chap. 17. 

Replacement of Items that Deteriorate by Different Equipment. 
Here, one considers the replacement of equipment by new or alternate 
pieces of equipment other than those currently in use. 

Let K' n = minimum discounted value of all future costs of new 

equipment, 
Di, D 2 , • • •, D m = costs in each future period incurred with present 
equipment, 
X = 1/(1 + r), the discount factor, 

7r m = discount value of all future costs if present equipment 
is discarded after m periods. 
Cost Equation. 

(99) tt w = D x + D 2 X + • • • + D m X m ~ l + K' n X m . 
Therefore 

(100) 7T m+1 -T m = D m+1 X m + K' n (X m+1 - X m ), 

and 

(101) 7T W - 7r m _! = D n X m ~ l + K' n (X m - I"" 1 ). 
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The condition 7r TO _i — 7r m > is equivalent to 

(102) D m < (1 - X)K' n 
whereas the condition 7r OT+ i — T m > is equivalent to 

(103) D m+1 >(1 - X)K' n . 

Conditions (102) and (103) show that the minimum cost is achieved by 
continuing the use of the old equipment until the cost for the next period is 
greater than (1 — X)K' n , where (1 — X)K' n is the weighted average of the 
costs of using the equipment for n periods between replacements. 

Replacement of Items That Fail 

The second class of replacement problems is concerned with items that 
do not deteriorate markedly with service but which ultimately fail after a 
period of use. The period between installation and failure is not constant 
for any particular type of equipment but will follow some frequency distri- 
bution. This section is concerned only with items that fail with increasing 
probability as they age. Furthermore, it is assumed hereafter that all fail- 
ures will be replaced. The problem, therefore, is to plan the replacement 
of items that have not failed. 

Replacing a used but still functioning item with a new item is justified 
only if the cost of replacement is higher after failure than before, and if 
installing the new item reduces the probability of failure. 

The replacement policy will depend upon the probability of failure. It 
is therefore of considerable importance to estimate the probability distri- 
bution of failures. Statistical techniques used in such "life testing" are 
being developed rapidly and a growing literature on the subject is becom- 
ing available. See Refs. 63-65. The costs of replacement before and after 
failure are the other important factors. 

In this section, the cost of the alternatives of replacement or retention is 
considered and two policies are developed that minimize expected costs as 
a function of the cost of replacement, cost of failure, and probability of 
failure. 

Mortality Curves. The initial information on the life characteristics 
of a light bulb, for example, may be shown in the form of a mortality curve. 
A group of N light bulbs is installed, and at the end of t equal time intervals 
the number of bulbs surviving equals some function of t, say S(t). The 
proportion of the initial bulbs remaining is, then, 

(104) s(t) = -^ 

N 
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A typical mortality table is shown in Table 34 giving, at regular intervals 
of time, the number of survivors out of an original group of 100,000 bulbs. 
Specifically, the mortality curve would result from column 2 in Table 34, 
namely that given by S(t), and is given in Fig. 10. 



Table 34. 


Life Charac: 


(1) 


(2) 


Time 




Units 




Elapsed 


Survivors 


t 


SQL) 





100,000 


1 


100,000 


2 


99,000 


3 


98,000 


4 


97,000 


5 


96,000 


6 


93,000 


7 


87,000 


8 


77,000 


9 


63,000 


10 


48,000 


11 


32,000 


12 


18,000 


13 


10,000 


14 


6,000 


15 


3,000 


16 


2,000 


17 


1,000 


18 






Original Population 



stics of a Light Bulb: 
of 100,000 Units 

(3) 

Reduction in 

Survivors 

S(t - 1) - S(t) 





1,000 

1,000 

1,000 

1,000 

3,000 

6,000 
10,000 
14,000 
15,000 
16,000 
14,000 

8,000 

4,000 

3,000 

1,000 

1,000 

1,000 

Column (1), number of elapsed periods. 

Column (2), survivors at end of period, based on figures supplied by a major light 
bulb manufacturer. 

Column (3), rate of change of column (2). 

Column (4), column (3) divided by 100,000. 

Column (5), column (3) divided by value in column (2) for previous period. 

Life Span. Perhaps a more familiar presentation of the life charac- 
teristics of a group of items is in the form of a probability distribution of 
life spans. Such a probability distribution may be derived from the mor- 
tality table by taking 

S(t - 1) - S(t) 
(105) = p(0, 



(4) 


?). 




Conditional 


)bability 


Probability of 


Failure 


Failure 


P(0 


Vtfi 








0.01 


0.0100 


0.01 


0.0101 


0.01 


0.0102 


0.01 


0.0103 


0.03 


0.0312 


0.06 


0.0645 


0.10 


0.1149 


0.14 


0.1818 


0.15 


0.2381 


0.16 


0.3333 


0.14 


0.4375 


0.08 


0.4444 


0.04 


0.4000 


0.03 


0.5000 


0.01 


0.3333 


0.01 


0.5000 


0.01 


1.0000 



N 

the proportion of units failing in time period t. (See Table 34.) 
probability function, p(t), is plotted against t in Fig. 11. 
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2 4 6 8 10 12 14 16 18 
Time units elapsed, t 

Fig. 10. Number of survivors after t periods of time. (Data from Table 34.) 
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Fig. 11. Probability of failure in tth. period of bulb installed at beginning of first period. 

(Data from Table 34.) 
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Conditional Probability of Failure. Another descriptive notion of 
life characteristics is the conditional probability of failure or its comple- 
ment, the probability that an item at time t will survive to time t + 1. 
This probability is given by 



(106) 



8{t-l)-8{t) S(t) 
v t0 = =1 

S(t - 1) S(t - 1) 



and is the proportion of surviving units failing in the subsequent period. 
(See Table 34.) This conditional probability function is plotted against t 
in Fig. 12. 

1.0 r- 




6 8 10 12 
Time units elapsed, t 

Fig. 12. Conditional probability of failure in tih. period. (Data from Table 34.) 

Replacement Process. It is assumed here that failures occur only at 
the end of a unit period of time. During the first t — 1 time intervals, all 
failures occurring during any given time interval are replaced at the begin- 
ning of the next time interval. At the end of the tth. time interval, all units 
are replaced regardless of their ages. The problem is to determine that 
value of t which will minimize total cost. 

Rate of Replacement. The general expression for the number of units 
failing in time interval t is 



(107) f(t) 



I x=l 6=2 



t— 1 rb—1 

Z) p(*)pQ> - x ) 

=2 Lz=l 



P(t~b) + 



where N = total units in the installation, 
j)(x) = probability of failure at age x. 



OPERATIONS RESEARCH 



15-93 



Table 35 illustrates the use of eq. (107) to determine the total number of 
failures in each time period t, based upon the data of Table 34. 

Table 35. Total Failures (Replacements) in Each Period t a 



(1) 


(2) 


(3) 


(1) 


(2) 


(3) 




Replacements 




Replacements 




Current 


Cumulative 


Current 


Cumulative 


eriod 


/(0 


2/(0 


Period 


/(0 


2/(0 


1 








21 


12,047 


162,167 


2 


1,000 


1,000 


22 


11,706 


173,873 


3 


1,000 


2,000 


23 


10,820 


184,693 


4 


1,010 


3,010 


24 


9,697 


194,390 


5 


1,020 


4,030 


25 


8,700 


203,090 


6 


3,030 


7,060 


26 


8,288 


211,378 


7 


6,040 


13,100 


27 


8,413 


219,791 


8 


10,090 


23,190 


28 


8,862 


228,653 


9 


14,201 


37,391 


29 


9,523 


238,176 


10 


15,392 


52,783 


30 


10,100 


248,276 


11 


16,665 


69,448 


31 


10,413 


258,689 


12 


15,000 


84,448 


32 


10,507 


269,196 


13 


9,480 


93,928 


33 


10,348 


279,544 


14 


6,175 


100,103 


34 


9,999 


289,543 


15 


6,160 


106,263 


35 


9,636 


299,179 


16 


5,521 


111,784 


36 


9,079 


308,258 


17 


7,309 


119,093 


37 


9,220 


317,478 


18 


9,317 


128,410 


38 


9,271 


326,749 


19 


10,181 


138,591 


39 


9,447 


336,196 


20 


11,529 


150,120 


40 


9,669 


345,865 



Column (1), periods since original installation. 
Column (2), calculated as described in text. 
Column (3), cumulative sum of values in column (2). 

° Data based on Table 34. 



A second method for determining the number of failures in each period t, 
based upon the conditional probability of failure and using vector algebra 
is given in Ref. 1, Chap. 17. 

Cost of Replacement. A second fundamental requirement of a useful 
replacement policy is that the cost of replacement after failure be greater 
than the cost of replacement before failure. This difference in cost is the 
source of savings required to compensate for the expense of reducing the 
probability of failure by replacing surviving units. Group replacing of units 
can cost less than replacement of failures by virtue of labor savings, volume 
discounts on materials, or for other reasons. 
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Cost Equation. Let K(t) = total cost from time of group installation 
until the end of t periods. 

Then, if the entire group is replaced at intervals of length t periods, 

K(t) 

= Average cost per period of time. 

Let Ci = unit cost of replacement in a group, 

C 2 = unit cost of individual replacement after failure, 
f(X) = number of failures in the Xth period, 
N = number of units in the group. . 
Then the total cost, K(t), will be given by . 

t— 1 

(108) K(t) = Nd + C 2 S f(X). 

x=i 

Therefore, the cost per period is given by 

K(t) Nd C 2 '- 1 

(109) — = + — E /(X). 

t t t x=\ 

Minimization of Costs. Costs are minimized for a policy of group replac-- 
ing after t periods if 

K(t) K(t - 1) 

(110) — - < — -, 

t t- 1 

and if 

K(t) K(t + 1) 

(111) -^ < — - 

t t+ 1 

By using eq. (109), conditions (110) and (111) maybe rewritten respectively 

as 

t— 2 

Nd + C 2 £ f(X) 

(112) C 2 f(t - 1)< j-^ . 

and 

t— l 

Nd + d S f(X) 

(113) C 2 f(t) > — 

Conditions (110) and (111) and, in turn, conditions (112) and (113), are 
necessary conditions for optimum group replacement. They are not suffi- 
cient as illustrated by the function F(t) = t sin t, ^ t fS 47r, which satisfies 
these conditions for not one but two values of t, although the function has 
but one true (as opposed to relative) minimum point. 



Table 36. Average Costs for Alternative Group Replacement Policies: Ci/C 2 = 0.25 



(1) 



(2) 



(3) 



(Data from Table 35) 
(4) (5) 



(6) 







t 








Total Cost 




fit) 
(Current) 


Z/P0 




t-2 


t-1 


t-i 


t 


(Cumulative) 


(t - V)f(t - 1) - 


Zf(X) 
1 


W) - Zf(X) 


K(t) =NC 1 + C 2 Zf(X) 
i 


1 
















25,000C 2 


2 


1,000 


1,000 







2,000 


25,000C 2 


3 


1,000 


2,000 


2,000 




2,000 


26,000C 2 


4 


1,010 


3,010 


2,000 




2,040 


27,000C 2 


5 


1,020 


4,030 


2,040 




2,090 


28,010C 2 


6 


3,030 


7,060 


2,090 




14,150 


29,030C 2 


7 


6,040 


13,100 


14,150 




35,220 


32,060C 2 


8 


10,090 


23,190 


35,220 




67,620 


38,100C 2 


9 


14,201 


37,391 


67,620 




104,619 


48,190C 2 


10 


15,392 


52,783 


104,619 




116,529 


62,391 C 2 



Column (1), number of periods between group replacements. 
Column (2), number of replacements from Table 35. 
Columns (3), (4), (5), calculated as indicated in column headings 
Column (6), calculated as indicated with C\ = 0.25C 2 . 
Column (7), column (6) divided by column (1). 

° Therefore i = 7. 



(7) 




Average 




Cost per 




Period 




K(t)/t 


O 


25,000C 2 


-o 
rn 


12,500(7 2 


TO 

> 


8,667C 2 


o 

z 


6,750C 2 


5,602C 2 


CO 


4,838C 2 


TO 

m 


4,580C 2 « 


to 
m 


4,762C 2 


> 

•JO 


5,354C 2 


n 


6,239C 2 


X 
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Conditions (112) and (113) may be interpreted as follows: 

1. One should not group replace at the end of the tth period if the cost of 
individual replacements at the end of the tth period is less than the average 
cost per period through the end of t periods. 

2. One should group replace at the end of the tth period if the cost of 
individual replacements for the tth period is greater than the average cost 
per period through the end of t periods. 

The use of these decision rules for the light bulb example (see Table 34) 
is illustrated in Table 36. For a full discussion of this replacement model 
and the solution of the light bulb example, see Ref. 1. 

Solution of Replacement Problems by Monte Carlo Technique 

In the determination of optimum group replacement policies for items 
that fail, one needs to determine that value of t such that 

(114) K(t) = NCi + C 2 0(O 

will be a minimum, when 

N = number of units in the group, 
C\ = unit cost of replacement in a group, 
C2 = unit cost of individual replacement after failure, 
(f>(t) = number of failures in time t. 
Failure Equations. If fit) is the probability density function of failure, 
then the expected number of first generation failures in time t is given by 

(115) 4>(t) =n[ f(t)dt = NF(t), 
where 

(116) F(t) = ff(t)dt. 

Similarly, the number of second generation failures is given by 



02 (0 = NJ (j f(t) <U) (J f(t) dt) da. 



That is, 



(117) ' 4> 2 {t) = N f F(a)F(t - a) da. 

The number of third generation failures is given by 

03(0 =Nf f ( f f(t) dt) ( f fit) dt) ( f fit) dt) dfi da. 
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That is, 

(118) <j> 3 (t) = N f f F(a)F(P - a)F(t - 0) dp da. 

Therefore, the total expected number of failures in time t is given by 

That is, 

(119) <j>(t) = NF(t) + JV f ^(a)^(^ - a) da 

+ N I I F(a)F($ - a)F(t - fi) d$ da -\ . 

Unless simplifying assumptions are made relative to second and higher 
generation failures, it is almost impossible to obtain an analytic solution 
for <j>(t) as given by eq. (119). However, by the use of the Monte Carlo 
technique one can solve for values of </>(0 without making any simplifying 
assumptions. That is, one can determine <j>(t) for many values of t and then 
construct K(t) as a function of t in order to determine the optimum group 
replacement policy. 

Example. The Use of the Monte Carlo Technique in Solving Replacement 
Problems. Assume that one wishes to determine the optimum group re- 
placement policy for a group of light bulbs whose life pattern follows a 
normal distribution, the mean and standard deviations of which are 30 and 
10 days, respectively. (That is, n = 30 days and <r = 10 days.) Further- 
more, assume that 

Ci = $0.50, 

C 2 = $1.00, 

N = 10, 

T = 360 days, 

where T is the total time period under consideration. 

For purposes of illustration only, further assume that if group replace- 
ment is used, it can be done only at the end of 10, 20, 30, or 40 days. 

A chart can then be set up and, by use of a table of random normal num- 
bers, the total expected number of failures can be determined for each 
value of t (t — 10, 20, 30, or 40 days). Tables of random normal numbers 
are based on a mean of and a standard deviation of 1. Hence, any num- 
ber selected from the table of random normal numbers must first be multi- 
plied by 10 and then added to (if positive) or subtracted from (if negative) 
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30. Thus, if the first random normal number selected from the table is 
0.464, the adjusted random normal number will be 

(0.464) (10) + 30 = 34.64. 

That is, in the simulation of the light bulb system, the first bulb will last 
34.64 (or 35) days before failing. 

The next number from the table of random normal numbers is, say, 
0.137, which is adjusted to (0.137)(10) + 30 = 31.37. Therefore, the re- 
placement to the first bulb will last 31 days, that is, 35 + 31 = 66 days 
after the start of the analysis. Therefore one can expect the first bulb to 
burn out after 35 days and its replacement to last through the balance of 
the 40-day period under discussion. 

This procedure is carried out for all ten lighting fixtures in the installa- 
tion, and the expected number of failures for each of the intervals 10, 20, 
30, and 40 days is determined as in Table 37. 

Table 37. Failure Table for N = 10 



x. Fixture 






















Total 


t >v 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


Fail- 
ures 


10 


35 


31 


55 


27 


29 


33 


27 


43 


32 


20 





20 


35 


31 


55 


27 


29 


33 


27 


43 


32 


20 





30 


35 


31 


55 


27,38 


29,64 


33 


27,59 


43 


32 


20,55 


4 


40 


35,66 


31, 36, 62 


55 


27, 38, 75 


29,64 


33,47 


27,59 


43 


32,36 


20,55 


10 



The entire procedure was repeated nine more times and the ten samples 
(each of sample size N = 10) gave the results shown in Table 38. 



Table 



Summary Table for Ten Samples with N = 10 



t 

10 
20 
30 
40 



Total Number 
of Failures 

1 
15 
51 
96 



Average Number 
of Failures, <f>{t) 

0.1 
1.5 
5.1 
9.6 



From Table 38 and eq. (114), one can determine and compare the cost of 
group replacement for the 10-, 20-, 30-, and 40-day periods. These total 
expected costs over time period T (360 days) are : 



#io = WK10) (0.50) + (0.1) (1.00)] = $183.60, 
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K 20 = - 3 #[(10)(0.50) + (1.5)(1.00)] = $117.00, 
#30 = ^[(10)(0.50) 4- (5.1)(1.00)] = $121.20, 
/Co = -W[(10)(0-50) + (9.6) (1.00)] = $131.40. 

Thus, if one is to group replace at 10-, 20-, 30-, or 40-day intervals, one 
would do so every 20 days. (This assumes, of course, that, in practice, one 
has taken a sufficiently large sample of random normal numbers.) 

If one did not group replace, one could expect to replace each bulb, on 
the average, every 30 days. Accordingly, the total expected cost over time 
period T of not group replacing, call it K x , is 

/Co = 10(4^) (1.00) = $120. 

Therefore, under the assumptions of this illustration, one should group 
replace every 20 days (since K x > K 2 q). 

Other Models 

Although the solutions presented apply only to the particular model 
described earlier, models of other characteristics may be approached in the 
same way. For example, a model could be concerned with group replace- 
ment in which new bulbs are used for group replacement only, and used 
bulbs replace failures in between group replacements. A different model 
is needed when surviving bulbs are replaced at a fixed age, rather than at 
fixed intervals of time. The considerations of this chapter have been lim- 
ited to demonstrating an approach to two basic replacement problems, one 
involving deterioration, and the other involving probabilistic life spans of 
equipment. 

For a discussion of the models which have been developed and solutions 
obtained for various sets of assumptions about the conditions of the prob- 
lem, see Ref. 1, Chap. 17. For a useful review of equipment replacement 
rules from an industrial point of view, see Ref. 66. 

7. COMPETITIVE PROBLEMS 

Introduction 

A competitive problem is one in which the efficiency of one's decision is 
affected by the decisions of one's competitors. Such problems include, for 
example, competitive advertising for a relatively fixed market or bidding 
for a given set of contracts. 

Game Problems. The most publicized competitive problem in O.R. 
is the "game" as developed by the late John von Neumann and discussed 
in his Zur Theorie der Gesellshaftsspiele (Ref. 67) in 1928 and, jointly with 
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Oskar Morgenstern, in their Theory of Games and Economic Behavior (Ref . 
68) in 1944. 

For many decades, economists tended to take as their standard model 
for their science, the situation of Robinson Crusoe, marooned on an unin- 
habited island and concerned with behaving in such a manner as to max- 
imize the goods he could obtain from nature. It was generally felt that it 
would be possible to gain insight into the behavior of groups of individuals 
by starting with a detailed analysis of the behavior in this simplest possible 
case: the case of a single individual all alone and struggling against nature. 

This line of attack on economic problems, however, suffers from the defect 
that in going from a one-man society to even a two-man society, qualita- 
tively different situations arise which could not have been foreseen from 
the one-man case. Von Neumann was led to believe that group economics 
could more profitably be viewed as analogous to parlor games of strategy. 

Von Neumann's game is characterized by a fixed set of rules and a known 
number of competitors whose possible choices are also known. Further- 
more, the payoff for each combination of choices is also assumed to be 
known. The solution to von Neumann's game is obtained by a principle 
of conservatism called the minimax principle, namely one which will max- 
imize the minimum expected gain or minimize the maximum expected loss. 

Very little has been accomplished by way of applying the von Neumann 
theory of games. Military applications have been referred to but have 
not been made public. Several authors have explored the possibility of 
applying game theory to industrial problems, but they have not dealt with 
actual applications. What then is the significance and value of game 
theory? This can best be answered by quoting Williams' (Ref. 69, p. 217) 
succinct appraisal: 

While there are specific applications today, despite the current limitations of the 
theory, perhaps its greatest contribution so far has been an intangible one; the 
general orientation given to people who are faced with over-complex problems. 
Even though these problems are not strictly solvable — certainly at the moment 
and probably for the indefinite future — it helps to have a framework in which to 
work on them. The concept of a strategy, the distinction among players, the role 
of chance events, the notion of matrix representations of the payoffs, the concepts 
of pure and mixed strategies, and so on, give valuable orientation to persons who 
must think about complicated conflict situations.* 

Bidding Problems. A second type of competitive problem is one in 
which bidding takes place. Bidding problems differ from game problems 
in that: (a) the number of competitors is not usually known, (6) the num- 
ber of choices is not known (since one can bid over a large range), and (c) 
the payoffs are not usually known but, rather, are subject to estimation 

* Reprinted by permission from The Compleat Strategyst by J. D. Williams, copy- 
right 1954. McGraw-Hill Book Co. 
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(e.g., in bidding for mineral rights). Furthermore, in some bidding situa- 
tions (e.g., those in which one bids a dollar amount plus a percentage of 
the royalties), one may not be able to determine readily whether or not a 
given bid would have won or lost. 

Only a limited theory of bidding exists to date, although the concepts 
and techniques of statistical decision theory hold great promise in this area. 
A major research contribution has been made by Friedman (Ref. 70 and 
Ref. 1, Chap. 19). The number of applications of bidding theory has been 
very limited; however, in at least one instance, the results obtained have 
been spectacularly successful. Bidding models will not be discussed here. 
See Refs. 1 and 70. 

The Theory of Games 

Definitions. 

Game, a set of rules and conventions for playing. 

Play, a particular possible realization of the rules. 

Move, a point in a game at which one of the players selects an alternative 
from some set of alternatives. 

Choice, that particular alternative selected. 

Strategy, a player's predetermined method for making his choices during 
the play. 

Classification of Games. 

1. Players, the number of sets of opposing interests: (a) one-person, (6) 
two-person, (3) n-person (n > 2). 

2. Payment, (a) zero-sum game, a game in which the sum of the payoffs, 
counting winnings as positive and losses as negative, to all players is zero; 

(b) nonzero-sum game, a game in which the sum of the payoffs to all players 
is not zero. 

3. Number of moves: (a) finite, (b) infinite. 

4. Number of choices: (a) finite, (6) infinite. 

5. Amount of information regarding opponent's choices: (a) all, (b) part, 

(c) none. 

One-Person Games. One-person zero-sum games are trivial games 
which say "do nothing" since there is no gain to be made by the one par- 
ticipant in the game. One-person nonzero-sum games are the ordinary 
maximization and minimization problems solvable by calculus and other 
optimization techniques. Thus, in order to study the characteristic prop- 
erties of games of strategy, it is necessary to go to games which involve 
more than one player. The discussion here will center mainly on two- 
person zero-sum games. 

Two-Person, Zero-Sum Games. Analysis of the very simplest of 
games shows that there are two general kinds, which may be illustrated 
by two kinds of coin matching. 
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Single Strategy Games. Assume that one is matching dimes and quarters 
where, if both coins are the same, it is a standoff, but if the coins differ, the 
quarter takes the dime. In this game it is safest always to play a quarter, 
for then one can never lose, whereas one may lose by playing a dime. 
Such games in which each opponent will find it safest to stick to one 
strategy are called single strategy games. 

Mixed Strategy Games. The second general type of game may be illus- 
trated by the usual penny-matching situation in which each player chooses 
either heads or tails. If the coins match, the matcher wins; if they do not 
match, the matchee wins. In this case, if either player sticks to one strat- 
egy, he may consistently lose. The only safe way to play the game is to 
play heads or tails in a completely random manner, as, for instance, by 
nipping the penny in the air just before one plays it. Such games are called 
games of mixed strategy. 

Payoff Matrix. Games can have any number of strategies. In principle, 
once each player has chosen one of the sets of strategies available to him, 
it is possible to calculate the probable outcome of the game. The net pay- 
offs can then be arranged in a two-dimensional matrix, the payoff matrix. 
From the payoff matrix, one can then find whether the game is a single or 
a mixed strategy game and, if mixed, in what proportions to mix the 
playing. 

For further discussion of the definitions, classification of games, and 
examples of the construction of payoff matrices, see Ref. 71, Chap. 1. 

Single Strategy, Two-Person, Zero-Sum Games 

Minimax Principle. Consider the game whose payoff matrix with 
respect to player Pi is 

a ll a 12 ' " ' a ln 
Ct2i a 2 2 ' • • «2n 

A = 



If player P x chooses the number i (i.e., adopts the ith. strategy, i = 1, 2, 
• • •, m), he is certain to receive at least 

mina;y, j = 1, 2, • • •, n. 
j 

Since he can choose i as he pleases, he can, in particular, choose i so as to 
make min,a;.,- as large as possible. Thus player P x can choose i so as to 
receive at least 

max min a;y. 
i j 
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Similarly, player P 2 can choose j so as to make certain that he will receive 

at least . , N 

max mm ( — a#), 

3 i 

since for two-person, zero-sum games, the payoff matrix with respect to 
player P 2 will consist of elements (payments) which are the negative of the 
elements of matrix A. That is, player P 2 can choose j so as to make certain 
that he will receive at least 

— min max a# 

3 i 

or, equivalently, that player Pi will get at most 

min max a#. 

3 i 

Saddle Point. In summary, P x can guarantee that he will receive at least 

max min a# 

i j 

and P 2 can prevent Pi from receiving more than 
If 



mm max ai 

i i 



max mm a# = min max a# = aj y = u, 

i i i j 

Pi will settle for v and P 2 will settle for —v. Games for which the equation 
above holds are called games with a saddle point. More specifically, (i , j ) 
is called a saddle point and a; o y is called the value of the game for player P 1% 
Furthermore, the best strategy for player P x is i , and the best strategy for 
player P 2 is j . (See Ref. 71, Chap. 1.) 

It should be noted that a saddle point of a matrix is a pair of integers 
(*o> Jo) sucn that ai j is at the same time the minimum element of its row 
and the maximum element of its column. 

Every single strategy two-person, zero-sum game has a saddle point. This 
saddle point provides the solution of the game by designating the best 
strategies for each player and the value of the game. Example. The game 
represented by 



4 


3 


7 


6 


5 


2 


1 








1 


3 


4 


2 


2 


1 


5 



has a saddle point at (1, 2). Its solution consists of the strategies 1 and 2, 
respectively, and the value of the game is 3. 
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Stated in another manner, every game which contains a saddle point is a 
single strategy game (see Ref. 71). Games without saddle points, such as 
the game represented by 

1 -1 

-1 1 
are mixed strategy games. 

Mixed Strategy, Two-Person, Zero-Sum Games 

Consider the (penny-matching) game whose payoff matrix is 



Pi 



Heads 



Tails 



r 

Heads 


2 

Tails 


+ 1 


-1 


-1 


+ 1 



Such a matrix has no saddle point and, hence, is not a single strategy game. 
Furthermore, one can readily see that it makes little difference to player Pi 
whether he chooses strategy 1 (heads) or strategy 2 (tails), for, in either 
case, he will receive 1 or — 1 according as P 2 makes the same or opposite 
choice. Player Pi must play the game by making his selections by means 
of some chance device. The procedures for determining optimum mixed 
strategies are discussed below. 

Dominance. If a = (cti, a 2 , • • • , a n ) and b = (b i} b 2 , • • •, b n ) are vectors 
(or rows or columns of a matrix), and if ai ^ b{ (for i = 1, 2, • • •, n), one 
says that a dominates b. If a z - > bi (for i = 1, 2, • • •, n), one says that a 
strictly dominates b. 

Convex Linear Combination. 



Let x 



(i) _ f„(» 



(V 



.(2) _ 



= (Xi 



(2) 



.(r) _ 



= (Xl 



(r) 



X = (x lf 



(1) 



), 



■ X (2) ) 



j ^n J, 
j X n ). 



Let 



a = (a h •••, a r ) 



such that ai ^ (i = 1, 2, • • •, r) and a\ -\- a 2 -\ \- a r = 1. Then x 

is a convex linear combination of rc (1) , • • •, # (r) with weights a\, • • - , a r if 



xj = aix/ 1 ^ + a 2 Xj^ H (- a r Xj^ , for-j = 1, 2, 
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Thus, the point (0, 15) is a convex linear combination (with weights ^, 
34 and y 2 ) of the points (6, 12), (-9, 15), and (4, 16). 

Theorem. Let Y be a rectangular game whose matrix is A ; suppose that, 
for some i, the i-th row of A is dominated by some convex linear combination of 
the other rows of A; let A' be the matrix obtained from A by omitting the i-th 
row; and let Y' be the rectangular game whose matrix is A'. Then the value of 
Y' is the same as the value of Y; every optimum strategy for P 2 in V is also 
an optimum strategy for P 2 in Y; and if w is any optimum strategy for Pi in 
Y' and x is the i-place extension of w, then x is an optimum strategy for Pi 
in Y. Moreover, if the i-th row of A is strictly dominated by the convex linear 
combination of the other rows of A, then every solution of Y can be obtained in 
this way from a solution of Y f . (See Ref. 71, Chap. 2.) 

Note. A similar theorem applies to dominating columns. (See Ref. 71, 
Chap. 2.) 

Example. The following example of the application of this theorem is 
cited in Ref. 71 (p. 50) : 



3 


2 


4 





3 


4 


2 


4 


4 


2 


4 








4 





8 



Row 1 is dominated by row 3, yielding 



3 4 


2 


4 


4 2 


4 





4 





8 



Column 1 dominates column 3, resulting in 



4 2 4 
2 4 
4 8 



Column 1 dominates a convex linear combination of columns 2 and 3, 
namely : 

4 > i(2) + i(4), 

2 = i(4) + J(0), 

4 = i(0) + i(8). 



15-106 



OPERATIONS RESEARCH 



Thus, the first column can be omitted, yielding 

2 4 

4 

8 

Row 1 is now dominated by a convex linear combination of rows 2 and 3, 
since 

2 = i(4) + J(0), 

4 = J(0) + i(8). 

Therefore, the matrix reduces to 

4 

8 

As will be seen later, the solution to this latter matrix consists of the mixed 
strategy (%, }/£) for each player and a game value of %. Therefore, the 
value of the original game is %, and the optimum strategy for the original 
game is (0, 0, %, }£) for each player. 

General Theorems for Rectangular Games (Refs. 1 and 71) 

Theorem 1. Every rectangular game has a specific value g. This value is 
unique. Furthermore, there exists for player P x a best strategy, i.e., there exist 
non-negative frequencies x\, x 2 , • • • , x m such that Xi + x 2 + • • •+ x m = 1 
and such that if he plays plan I with frequency x\, plan II with frequency 
x 2 , • ■ • , plan M with frequency x m , then he can assure himself at least an 
expected gain of g, which is the value of the game. 

Similarly, for player P 2 , there exists a best strategy Y = (y 1} y 2 , • • • , y n ), 

2/i + 2/2 H V Vn = 1, such that if P 2 played plans I, II, • • •, N with the 

above frequencies, respectively, he (P 2 ) can assure himself at most a loss of g. 

Theorem 2. The unknowns, x lf x 2 , • • •, x m , y\,y 2 , • • -, y m and g (for the 
solution of a game) can be determined from the following relations: 

m 

xi-\r x 2 -\ h x m ss X) x i = ' !i x i ^ (f = 1, 2, • • • , m) ; 

i=l 



y\ 4- V2 + 



+ y n = X) Vi = l, 

i=i 



X) x i a ii ^ 9 
i=l 

n 

Z) a iiVj ^ 9 

3=1 



Vi ^0 (j= 1, 2, 
(j = l,2, ...,n); 



(i = 1,2, • ••, m). 



'7*); 
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Theorem 3. LetX* = (x x * } x 2 *, • • -,x m *) and Y* = (?/i*, i/ 2 *, • • •, y n *) 
be any optimal strategies for P 1 and P 2 , respectively, for a game whose value is g. 
If, for any i, 

X) aijijj < g, 



then 
Similarly, if for any j, 

then 



= 0. 



2 x t Hij > g, 



Vi* = 0. 



Solutions of Rectangular Games 

Two-by-Two Games. To solve two-by-two rectangular games, first 
look for a saddle point. If one exists, the game is a single strategy game and 
the solution is immediately given as discussed above. If no saddle point 
exists, the game is a mixed strategy game and is solvable by either of the 
following methods. 

Algebraic Solution. 

Given : 



a 
c 


b 
d 



Let x and 1 — x be the frequencies with which Pi plays plans I and II 
respectively. Then, if player P 2 plays plan I, Pi can expect 

a(x) + c(l — x) = c -f- (a — c)x. 

On the other hand, if player P 2 plays plan II, Pi can expect 

b(x) + d(l - x) = d + (6 - d)x. 

The solution of any two-by-two game is given by the minimax principle, 
namely, by solving 

c -\- (a — c)x = d + (6 — d)x. 

Example. Given: 



-3 


7 


6 


1 
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-3(x) + 6(1 - x) = 7(x) + 1(1 - x) 



x ~ 3) 



(l-x)= %. 



Similarly, one determines that 



77 - -2- 

y — 5> 



and (1 - y) = f . 

Finally, the value of the game is given by (since x = }/£) 
g = -3(|) + 6(f) = 3. 

For this and other algebraic procedures, see Ref. 1, Chap. 18. 

Method of Oddments (Two-by-Two Game). 

The method of oddments for two-by-two games is given by Williams 
(Ref. 69). 

Example. The method may be stated by means of the game whose 
payoff matrix is 

P 2 



Plan 


I 


II 


I 


-3 


7 


II 


6 


1 



To determine the optimum frequencies for P\, subtract the numbers in 
the second column from those in the first column. This gives: 



-10 



One of the two numbers will always be negative. Ignore the minus sign 
for the purpose of computing oddments. 
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whereas the oddment for Pi (II) is given by 



II 



10 



Therefore, the oddments for P x are 5 and 10, respectively, or, equiva- 
lently, the optimum frequencies are 

5 1 10 2 

= - and = -• 

5+10 3 5+10 3 

Similarly, by subtracting rows, one can determine that the optimum 
frequencies for player P 2 are % and %. 

Two-by-ra Games. To find the solution of a two-by-n game : 

1. Look for a saddle point. If one exists, the game is a single strategy 
game and the solution is given by the saddle point. 

2. If no. saddle point exists, examine the payoff matrix for dominance 
and, eliminate all dominated strategies (if any) for Pi and all dominant 
strategies (if any) for P 2 . 

3. The matrix which remains will then contain a two-by-two submatrix 
with the property that its solution is also a solution to the two-by-n game. 
The pertinent two-by-two submatrix can be found in one of several ways, 
probably the easiest of which is the graphical method. 

Graphical Solution of Two-by-n Games. 
Given the game whose payoff matrix is : 

(P 2 ) 



(Pi) 





1 


2 


3 


4 


5 


6 


7 


1 


-6 


1 


3 


5 





-4 


-1 


2 


7 


3 


-2 


4 


-3 





1 
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Plot the payoffs for each strategy of P 2 on two parallel axes, as shown in 
Fig. 13. 

Then, join the line segments which bound the figure from below and mark 
the highest point on this boundary. The lines which intersect at this. point 
identify the strategies that player P 2 should use. 




Fig. 13. Graphical solution of two-by-n game. 

In the given example, these are strategies 5 and 6. (Note that strategies 
2, 4, and 7 dominate strategy 6 and could have been eliminated immedi- 
ately. Similarly strategy 3 dominates 5 and could be eliminated.) There- 
fore, the appropriate two-by-two subgame is 

(P 2 ) 
5 6 



(Pi) 



which, by the method of oddments, gives 

x* = \f, i) and y* = (f, f) 
and a game value of g = — Y %. 






-4 


-3 
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Hence, the solution to the original game is 

X* = (I j-) and Y* = (0, 0, 0, 0, f , f 0), 
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and, again, g 



— —12, 



f 



It should be noted that, for ?n-by-two games, one proceeds as above, 
marking, however, the line segments which bounds the graph from above 
and then identifying the lowest point on this boundary. Note. This is 
merely a graphical application of the minimax principle. See Refs. 1, 69, 
and 71. 

Three-by-Three Games. To find the solution of three-by-three 
games : 

1. Look for saddle point. 

2. If none exists, examine the payoff matrix for dominance and reduce it 
accordingly. 

3. If a three-by-three matrix remains, solve by method of oddments to 
see if a three-by-three solution exists. 

4. If oddments method fails, try the two-by-two subgames for a solution. 
Example. Method of Oddments (Three-by-Three Game). Consider the 

game whose payoff matrix is 

CP 2 ) 

1 2 3 



(Pi) 2 
3 



To determine the optimum frequencies for player P lf subtract each column 
from the preceding column, yielding 



6 





6 


8 


-2 





4 


6 


5 



(Pi) 2 
3 



6 


-6 


10 


-2 


-2 


1 
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The oddment for Pi(l) is given by 

1 







10 


-2 


-2 


1 



the numerical value of which is the difference between the diagonal prod- 
ucts: 

10(1) - (-2)(-2) = 6. 

Similarly, Pi (2) is given by 



namely, 



and Pi (3) is given by 





6 


-6 


2 








-2 


1 


6(1)- 


(-6)(-2) = 




6 


-6 




10 


-2 


3 







-6, 



or 

(6) (-2) - (-6) (10) = 48. 

Therefore, the oddments for P x are 

6:6:48 

so that the optimum frequencies are 

■*■ — UO) 10> 5)' 
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Similarly, by subtracting rows, one determines the oddments for P 2 , 
namely, 

38:14:8 
and optimum frequencies 

V* — /" l 9 7 _4\ 
1 — V30> 30) 30/- 

Furthermore, the value of the game is given by 
(1)(6) + 1(8) + 8(4) 23 



g = 



10 



Note. Every solution obtained by the method of oddments must be 
tested. It may well be that the three-by-three game does not have a three- 
by-three solution, but, rather, a two-by-two solution. See Ref. 69. 

Three-by-n Games. To find the solution of three-by-n games : 

1. Look for a saddle point. 

2. If none exists, examine the payoff matrix for dominance and reduce it 
accordingly. 

3. If a three-by-n matrix remains, the problem is then to find the solution 
by the earlier methods, since every three-by-n matrix has solutions which 
are either three-by-three or two-by-two (or a saddle point). 

Solve the two-by-n subgames by the graphical method. If no two-by- 
two solutions exist, the solution must then be a three-by-three solution 
which can be obtained by successively trying each three-by-three subgame. 

See Refs. 1, 69, and 71. 

Four-by-Four Games. For games which do not have a saddle point 
(i.e., mixed strategy games) and which, after removing rows and/or col- 
umns due to dominance, reduce to a four-by-four game, there is a method 
of oddments for obtaining the desired solution. For this method, see 
Williams (Ref. 69). 

Other Solutions of Rectangular Games 

There are a variety of other methods for solving rectangular games, a 
few of which are cited here. 
Matrix Solution of Games. 

Let A = (a,ij) be the m X n matrix of a game, 

B = (bij) be any square submatrix of A of order r > 1, 

J r = (1, 1, • • •, 1), a 1 X r matrix, 

C T = transpose of C, where C is any matrix, 

adj B = adjoint of B, 

^ = \ x l> x 2t ' ' ') x m)i x i = Oj £/Xi = 1, 

Y = (j/i, y 2 , "-, Vn), Vj ^ 0, Xy 3 - = 1, 
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I = alXi" matrix obtained from X by deleting those elements 

corresponding to the rows deleted from A to obtain B, 
Y = a 1 X r matrix obtained from Y by deleting those elements 
corresponding to the columns deleted from A to obtain B. 
Solution. 

1. Choose a square submatrix B of A of order r (^2) and calculate 

_ J r adj B 

J r (adj B)J r 
and 

J r (adj £) r 

J r (adj B) J r x 

2. If some X{ < or some ijj < 0, reject the chosen B and try another. 

3. If x { ^ and y 3 - ^ for all i, j = 1, 2, • • •, r, calculate 

151 



J r (adj B)J r T 



and construct X and Y from X and Y by adding zeros in the appropriate 
places. 



Check whether 



and whether 



2 x i a H ^ 9, f or a11 J> 



i=i 



52 2/^7 ^ 9, for all f. 

/=i 

If one of the relations does not hold, try another B. If all relations hold, 
then X, Y, and g are the required solutions. See Refs. 1 and 71. 

Iterative Method for Solving a Game. There is an approximate 
method of solving rectangular games which enables one to find the value of 
such games to any desired degree of accuracy and also to approximate to 
optimal strategies. See Ref. 71, Chap. 4, and Ref. 1, Chap. 18. 

Solution of Rectangular Games by Linear Programming. It can 
be shown that the problem of solving an arbitrary rectangular game can 
be regarded as a special linear programming problem and, conversely, that 
many linear programming problems can be reduced to problems in game 
theory. Thus, the techniques for solving linear programming problems (e.g., 
the simplex technique), especially through the use of high-speed electronic 
computers, can be applied to the solution of game theory problems. See Ref. 71, 
Chap. 14. 
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Zero-Sum, n--Person Games 

The theory of w-person games, n > 2, is not in an altogether satisfactory- 
state. For an excellent exposition on the elements of zero-sum ?i-person 
games, see Ref. 71 and the original text on the subject, namely that of von 
Neumann and Morgenstern (Ref. 68). A very brief discussion is to be 
found in Ref. 1. 

8. DATA FOR MODEL TESTING 

Introduction. The type of evidence one uses to test a model depends 
very much on the kind of test one has in mind. In testing a model one 
asks, "What are the possible ways in which a model can fail to represent 
reality adequately and hence lose some of its potential usefulness?" Fol- 
lowing are four ways in which one may question the adequacy of a model. 

1. The model may assert a dependence of the effectiveness of the system 
(the dependent variable) on one or more (independent) variables which, 
as a matter of fact, do not affect the system's effectiveness. That is, the 
model may fail by including variables which are not pertinent. 

2. The model may jail to include a variable which does have a significant 
effect on the system's effectiveness. 

3. The model may inaccurately express the actual relationship which 
exists between the measure of effectiveness and one or more of the pertinent 
independent variables. 

4. Finally, even if the model is an accurate picture of reality in the sense 
of conforming to the foregoing three conditions, it may still fail to yield 
good results if the parameters contained in it are not evaluated properly. 

In testing the model, begin by testing it as a whole, i.e., by determining 
the accuracy of its prospective or retrospective predictions of the system's 
effectiveness. If this procedure shows that the model is not adequate, fur- 
ther testing will be required to find out which of the four types of deficiencies 
mentioned here is present. 

The design of the process of collecting data consists of the following 
parts: (1) definition (including measurement), (2) sampling (including ex- 
perimental designs), (3) data reduction, (4) use of the data in the test, (5) 
examination of the result, and (6) possible redesign of the evidence. 

Scientific Definitions. Scientific defining consists of specifying the 
best conceivable (not necessarily obtainable) conditions under which, and 
procedures by which, values of the variables can be obtained. 

Concern with ideal (or optimum) observational conditions and procedures 
is quite important if one wants to know how good are the results one even- 
tually obtains. Further, and more important, the ideal conditions and pro- 
cedures act as a standard by means of which one can evaluate the attainable 
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observational conditions and operations, determine their shortcomings, 
and make any necessary adjustments in the resultant data. For a detailed 
discussion of scientific defining, see Ref. 72. 

The two most common types of quantitative variables are the enumera- 
tive and the metric. The enumerative variable requires counting for its 
evaluation whereas the metric variable requires measurement. 

Scientific Definitions of Enumerative Variables. Two types of 
errors can arise in the counting operation, overenumeration and underenu- 
meration. Overenumeration results either from counting the same unit 
more than once or from counting units which should not be counted at all. 
Underenumeration, on the other hand, results from the failure to count a 
unit which should be counted. Furthermore, these errors can occur be- 
cause of a failure to match elements with consecutive integers (e.g., over- 
enumeration because of skipping numbers and underenumeration through 
duplication of numbers). 

It is desirable to design the best conceivable counting procedure, even if 
the design cannot be carried out in practice. This involves specifying the 
standard environment in which, and the standard operations by which, 
the count can ideally be made, as well as providing an explicit definition 
of the elements to be counted. Once this standard is specified, it will be 
possible to use it to evaluate alternative practically realizable counting 
procedures and to select the best of these. The standard also provides a 
basis for estimating the error that is likely to occur in the practical count- 
ing procedure which is eventually used. 

Scientific Definitions of Properties (Metric Variables). The ide- 
alized design of a procedure for measuring properties depends primarily on 
the type of property involved. Scientific definitions of properties involve 
specifying the following characteristics of the idealized measuring proce- 
dure: 

1. Identification of the thing, event, or class of things or events which 
should be observed. 

2. Specification of the environment in which the observations should be 
made. 

3. Specification of the changes in the environment which should be made, 
if any, during the observation period. 

4. Specification of the operations to be performed and the instruments 
and measure to be used by the observer. 

5. Specification of the readings (data) to be made. 

6. Specification of the analysis of the data. 

The formal description of the measure to be used states what logical and 
mathematical operations one wants to be able to perform on the data to be 
obtained in evaluating a variable. For a complete discussion of the theory 
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of measurement see Refs. 73-79. The scientific definition (observational 
standard) states how, ideally, one would go about collecting pertinent data. 
The operational specification of the data collection process states how one 
actually intends to collect and adjust the data. Errors can arise in each of 
these three stages of planning relative to testing the model. 

Sampling. In evaluating variables, one is either involved in measuring 
the property of a single unit or in counting the members of, or measuring 
the properties of, a class of units (a population). The definition of a prop- 
erty of a single unit specifies the conditions under which the observation 
should be made. If these conditions can be met and observations can be 
made without error, only one observation is required. But if the condi- 
tions are not met, observations are subject to error which can only be esti- 
mated if two or more observations are made. How many observations to 
make, and where, are sampling questions. Since the standard conditions 
specified in the definition can seldom be met in practice, one must choose 
one of two courses: (1) An experimental design must be chosen which, by 
techniques such as the analyses of variance and covariance, makes it pos- 
sible to assess the magnitude of the deviations and ascribe them to specific 
environmental factors, or (2) observations must be made on a subset (of 
the population) which make it possible to draw inferences that are valid 
for the whole population with the least possible bias. The subject of sam- 
pling is concerned with the selection of appropriate subsets. 

In the main, sampling can be described as the selection of items from a 
population. The "population" of objects, events, environments, and stim- 
uli to be sampled should be specified in the definition of the variable being 
evaluated. The population represents all the possible data of the relevant 
kind that can be collected. 

Evaluation of Samples and Sample Estimates. The decision which 
must be made in designing a sampling procedure is concerned with the 
method of drawing the sample and the method of making estimates about 
the population from the sample. If a prescribed method is carried out cor- 
rectly, there are two opposing considerations: (1) the probability that the 
estimate made on the basis of the sample will actually deviate from the 
true population value by an amount greater than some amount x; (2) the 
cost of taking the sample. 

In the main, the probability of deviations will decrease with an increase 
in the sample size, but the cost of taking the sample will increase with an 
increase in sample size. 

Types of Sampling Designs. In unrestricted random sampling every 
possible sample has the same chance of being chosen. Restricted random 
sampling represents methods by which each possible sample does not have 
an equal probability of being drawn. But in each case where random 
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sampling is used scientifically, the probability of selecting any sample is 
known. 

All the various schemes for sampling are based on very simple, practical 
considerations. These are: 

1. Items of the population may fall into recognizable groups (e.g., in 
terms of location or dollar amounts on an invoice). If this is the case, il is 
reasonable to think in terms of sampling from these groups, because in 
general one reduces the variance of the estimates and (more important) one 
can be selective in the amount of sampling that is done in each group. In- 
voices with large dollar amounts are more important than ones with small 
dollar amounts; hence a larger sample of the more important items should 
be taken. 

2. Items of a population often fall into clusters (e.g., a shipment shown 
on an invoice; people in a house, block, or town; items in a warehouse). If 
one looks at some item in a cluster, one might just as well look at the rest 
of the items. Hence the cluster becomes the basis of sampling, not the 
original items. The use of clusters may increase the variance of the esti- 
mates but greatly decrease the costs of gathering the sample — the usual 
economic balancing problem. 

3. One does not have to plan completely in advance. One can let the 
sample information that comes in dictate how the next steps are to be taken. 

The following is a general classification of the principal types of sampling 
designs : 

I. Fixed sampling design. The sampling design is fixed and not subject to 
change in terms of sample data. 

A. Unrestricted random sampling. A random sample is selected from 
the whole population by either 

1. Simple random sampling. Assigning a different number to each 
element in the population and using random numbers to select 
the sample, or 

2. Systematic random sampling. Where a population is ordered, 
selecting a starting place at random and then selecting subse- 
quent elements at a fixed interval from the first and subsequent 
selections. 

Tables of random numbers can be found in Refs. 10 and 80-82. 
Details on the generation of such numbers can be found in Ref. 83. 

B. Restricted random sampling. The population is divided into sub- 
groups (and possible subsubgroups, etc.) and either some of these 
are selected and/or random samples from some or all of these are 
selected. 

1. Multistage random sampling. Random samples are drawn from 
subgroups which have themselves been selected (a) with equal 
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probability, or (6) with probability proportionate to the relative 
size of the subgroup, or some other criterion. 

2. Stratified random sa?npling. A random sample is drawn from 
every subgroup of the population. The size of the sample from 
the subgroups may be (a) independent of the size of the sub- 
groups (i.e., samples of equal size), (b) proportionate to the rela- 
tive size of the subgroup, or (c) proportionate to the relative size 
of the subgroup and the dispersion of the elements within it 
(optimum allocation). 

3. Cluster sampling. A random sample of subgroups is selected, all 
elements of which are included in the final sample. 

4. Stratified cluster sampling. A combination of B2 and B3, where 
more than two stages of sampling are involved. 

II. Sequential sampling. A small random sample is selected and analyzed, 
on the basis of which a decision is made as to whether or not to continue 
sampling and if so, how. The samples may be either 

A. In groups, as in double or multiple sampling, or 

B. Single items taken one at a time. 

For details on sequential sampling see Refs. 84-87. 

The aspect of sampling called experimental design usually refers to a 
sampling plan based on the variables in the model which is to be tested. 
Instead of keeping everything fixed except one variable, it is possible to 
design data collection systems at optimum locations of some of the variables 
of the model. This sampling method assumes that the variables of the 
model can be manipulated in reality — or at least in a realistic model. For 
information on various types of experimental designs, see Refs. 88-91. For 
comprehensive surveys of contemporary sampling theory see Refs. 72 and 
92-95. 

Reduction of Data. The observations made on the sampled items or 
in a sample of situations provide the raw data on the basis of which var- 
iables are assigned values and hence, provide the basis for testing all or 
part of the model. In many cases the data require collation, editing, cod- 
ing, punching, etc. Discussion of these phases of data processing can be 
found in Ref. 72, Chap. X. 

In general, the ultimate form to which the data must be transformed to 
be useful in the testing process will be either an estimate of the value of a 
parameter or an inferential "statistic" which describes a relationship be- 
tween two or more variables. For example, in testing a lot-size model the 
cost variables must first be evaluated in order to compute the total cost 
"predicted" by the model. Once these predictions are obtained they are 
compared with observed values in order to derive a "statistic" which can 
be used to determine whether or not the model predicts well. 
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For a discussion of data reduction and the problems of estimation and 
obtaining estimates of the variability of estimates, see Ref. 1, Chap. 20. 

For statistical tests of the significance or nonsignificance of a variable, see 
Refs. 96-105. These tests may make it possible to determine such matters 
as : (a) whether a variable should or should not be included, (b) whether the 
form of an analytic function is linear or some other type, (c) whether the 
form of a probability function is normal or some other type, (d) whether 
the model has failed to include a variable that ought to have been included. 

For further discussion of procedures for testing the adequacy of models 
and the solutions derived from them, see Ref. 74, Chap. 20. 

9. CONTROLLING THE SOLUTION 

Introduction. Many, if not most, O.R. projects deal with mangement 
decisions that are recurrent. Hence the solution must be used over and 
over again. But the systems which are dealt with in O.R. are seldom stable. 
Their structure is subject to change. Relationships between the variables, 
or system parameters, which define the system and the value of the parame- 
ters themselves are usually subject to change. 

In such situations the relationships and parameters used in the decision 
rule must be adjusted for changes in the system as they occur. Costs may 
change, the distribution of demand may change in some or all of its char- 
acteristics, and the relationships between variables may change over time. 
Hence the values of the relationships and parameters should be periodically 
reevaluated and the assumptions involved in the model (from which the 
decision rule is derived) should be reexamined periodically. That is, the 
solution must be controlled lest it lose some of its effectiveness because of 
changes in the system. 

Complete methodologies have not yet been developed for optimizing 
control procedures. Enough is known, however, to design procedures 
which are more likely to lead to success than either leaving this phase of 
the project to chance or relying on others (management or operating per- 
sonnel) to take care of them. For a complete discussion, see Ref. 1, Chap. 
21. 

Controlling the Solution. The effectiveness of a solution in an O.R. 
problem may be reduced by changes in either values of the parameters of 
the system or the relationships between them, or both. A previously in- 
significant parameter may become significant, or, conversely, a previously 
significant parameter may become insignificant. Changes in the values and 
functional relations of the parameters which remain significant may also 
affect the effectiveness of the solution. 

Not every change in a parameter or relationship is significant. In gen- 
eral terms, a change is significant if (1) adjustment of the solution for the 
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change results in an improvement in effectiveness and (2) the cost of mak- 
ing the adjustment and carrying it out does not offset the improvement in 
effectiveness. 

Design of a control system, then, consists of three steps: (1) listing the 
variables, parameters, and relationships that either are included in the solu- 
tion or should be if their values were to change; (2) development of a pro- 
cedure for detecting significant changes in each of the parameters and rela- 
tionships listed; (3) specification of action to be taken or adjustments to be 
made in the solution when a significant change occurs. 

The last two steps are interrelated because determination of the signifi- 
cance of a change (step 2) depends on the cost of making the adjustment 
specified in step (3). 

Control of Parameters. The first step in designing a control procedure 
involves listing all the variables and relationships which, if they were to 
change in value, might affect the effectiveness of the solution. 

The parameters which are listed should be classified into two types: 

1. Variables whose values during the period covered by a decision can 
be known in advance, such as the number of models in a line, the number 
of work days in an accounting period, and the price for which an item is to 
be sold. Control of such measures consists either (a) of establishing com- 
munication lines between those who know these values and those who use 
the decision rules or (b) of providing the latter with source material (such 
as a calendar in the case of the number of work days per accounting period). 

2. Measures whose values cannot be known in advance, such as number 
of units sold, number of hours worked, and arrival rate of trucks. These 
values must be estimated in advance. 

Essential to the control of any measure, is the determination of whether 
its true value or one or more of the characteristics of its estimate have 
changed. This determination consists of testing the hypothesis that no 
change has occurred in the variable or the characteristics of its estimate 
(which are themselves variables). 

Errors in Detecting Changes. Determination of whether or not such a 
change has occurred is subject to two types of error: (I) asserting that a 
change has occurred when it has not and (II) asserting that a change has 
not occurred when it has. An understanding of the two types of error is 
essential to comprehend what is involved in controlling a variable. See 
Ref. 1, Chap. 21. 

Detection of and Adjustment for Significant Changes. Ideally, in 
the design of a control system for a variable, six interdependent decisions 
should be made if possible. (In some situations there may be no choice 
with regard to one or more of these decisions.) The decisions are as 
follows : 
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1. The frequency of (i.e., period between) control checks. 

2. The number of observations per control check, if more than one is 
possible. 

3. The way items should be selected for observation (i.e., the sampling 
design), if more than one observation is specified. 

4. The statistical testing procedure to be used to determine whether or 
not a value has changed. 

5. The specific decision rule based on the test. 

6. The action to be taken, if the test indicates that a parameter's value 
has changed. 

Costs. Again, ideally, these decisions should be made in such a way as 
to minimize the sum of the following costs : 

1. The cost of taking the observations. 

2. The cost of performing the test. 

3. The expected cost of a type I error (i.e., the cost of changing a value 
when it is not warranted). 

4. The expected cost of type II errors (i.e., the cost of not changing a 
value when it is warranted). 

Unfortunately, at the present time the six decisions listed cannot be 
made in such a way as to assure minimization of the sum of the four costs. 
The design of an optimizing procedure can be specified in general terms; 
i.e., a model can be constructed which expresses the total expected cost as 
an abstract (but not as a concrete) function of the six decisions. In addi- 
tion, some of the expressions which would appear in the model cannot be 
evaluated. In most situations, for example, the expected costs associated 
with type I and type II errors cannot be determined. For further reading, 
see Refs. 106-109. 

Further details on methods of controlling parameters are given in Refs. 
110-118. 

Controlling Relationships. Every probability distribution asserts a 
relationship between the probability of an event and the values of other 
variables. In the case of distributions which appear in the model and solu- 
tion, as, for example, the distribution of demand, the parameters which 
define the distribution must be controlled (e.g., the mean and variance) as 
well as the form of the distribution (e.g., normal or Poisson). Both aspects 
of the distribution should be subjected to control. 

There are no "standard" procedures for controlling the form of a dis- 
tribution. Such control may be obtained by periodically testing the 
"goodness of fit" in the manner given by standard statistics texts. The 
frequency with which such tests should be conducted depends on the rate 
at which data are generated. The visual plotting of data, as they become 
available, can frequently indicate when a check should be made. Examina- 
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tion of these charts can provide clues to changes in the parameters of the 
distribution as well as to the form. 

The control of relationships which do not take the form of probability 
distributions also involves control over the form of the function which re- 
lates the variables and the values of the variables. 

Every O.R. project has unique characteristics which create unique con- 
trol problems but which also offer challenging opportunities for the develop- 
ment of unusual control procedures. There is a good deal of room for 
scientific creativity in this phase of the research. For a full discussion and 
illustration of the development of control procedures, see Ref. 1, Chap. 21. 

10. IMPLEMENTATION 

Concern of the O.R. Team. Once the solution has been derived and 
tested, it is ready to be put to work. Conversion of the solution into opera- 
tion should be of direct concern to the research team for two reasons: 

1. No matter how much care has been taken in deriving a decision rule 
and testing it, shortcomings may still appear when it is put into operation 
or ways of improving the solution may become apparent. If adjustment 
of the decision rule to take care of unforeseen operating problems is left in 
the hands of those who do not understand how it was derived, the adjust- 
ment may seriously reduce its effectiveness. Operating personnel may, 
for example, see no harm in making what appears to them to be a slight 
change, but such a change may be critical. 

2. Carrying out the solution may not be as obvious a procedure in the 
context of complex operations as it initially appears to be to the researchers. 
The solution must be translated into a procedure that is workable if its 
potential is to be fully realized. The procedure must be as accurate a 
translation of the solution as is practically feasible and only the researchers 
can minimize the loss in the solution's effectiveness that is incurred in this 
translation. 

The nature of the implementation problem depends on whether the solu- 
tion pertains to a one-time or repetitive decision. In the case of one-time 
decisions the problem is simpler but by no means disappears. 

Translation of the solution into the operating procedure involves answer- 
ing three questions and proceeding accordingly. The three questions are: 
(1) Who should do what? (2) When? (3) What information and facilities 
are required to do it? On the basis of the answers to these questions the 
operating procedure can be designed and any necessary training and transi- 
tion can be planned and executed. 

Implementation of a solution involves people taking action. These peo- 
ple must be identified, and the required action must be specified. The de- 
tails cannot be enumerated without a thorough knowledge of the opera- 
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tions and the division of responsibility in the organization under study. 
The analysis of the organization provides much of the needed information 
and the rest should be provided by management and operating people work- 
ing with the research team. This and the other phases of implementation 
require continuous cooperation and communication among management, 
operators, and researchers. 

Each person who is given responsibility for initiating action in carrying 
out the solution or using the decision rule should be instructed as to when 
they are to take action. The tools required to do the job should be made 
available to those who need them, and these people should be trained in 
their use. The tools should not be too complex for the operating personnel 
to use. It may be necessary, for example, to convert even a simple equation 
into a nomograph or tables. In some cases the tools may require simplifica- 
tion even if such simplification results in a loss of some of the original solu- 
tion's power. The solution or decision rule is generally used by personnel 
whose mathematical sophistication is less than desired. Consequently, if 
one wants to assure use of the recommended decision rules, one must fre- 
quently simplify them before handing them over to executives and operat- 
ing personnel. In many cases this means that one must either translate 
elegant solutions into approximations that are easy to use or sidestep the 
elegance and move directly to a "quick-and-dirty" decision rule. 

It should be realized that in one sense almost every solution in O.R. is an 
approximation and is "quick and dirty" to some degree. This follows from 
the fact that in constructing every model some simplifying assumptions 
are made. Reality is too difficult to represent in all its complexity. These 
simplifying assumptions reduce the generality of the model and solutions 
derived from it. But this is only a polite way of saying that quickness and 
dirtiness are involved. It is well for the operations researcher to realize 
that an approximate solution which is used may be a great deal better 
than a more exact solution which is not. 

For further discussion of the problems of implementing the solution, see 
Ref. 1, Chap. 21. 
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1. INTRODUCTION 

Basis of Information Theory. As used here, information theory is a 
body of results based on a particular quantitative definition of amount of 
information. This definition has a firm claim to unique importance in 
connection with the engineering questions which arise in systems which 
transmit and store information. It has proved interesting and sometimes 
useful in other fields (Refs. 1-4). However, other definitions have also 
been proposed (Ref. 5) and one of them has a long and useful history in 
statistics (Ref. 6). Caution is therefore needed in applying this definition 
to a situation in which the theorems which are its main justification in 
transmission and storage problems do not apply. 

Communication Theory. Information theory is a subdivision of a 
broader field, the statistical theory of communication, which includes all the 
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probabilistic analysis of communications problems. This broad field in- 
cludes in addition to information theory the analysis of random noise 
(Ref. 7), work on optimum linear filtering and prediction (Ref. 8, see also 
Chap. 17), statistical analysis of signal detection (Refs. 9, 10), and many 
other applications of probabilistic ideas which make no use of an informa- 
tion measure. 

Note on Terminology. Some authors, particularly in England, use 
information theory in a very broad sense, to include theories of scientific 
method and of statistical inference along with communications problems 
(Ref. 11). They then use "communication theory," or "mathematical 
theory of communication," or "theory of selective information," to denote 
what is here defined as information theory. 

Mathematical Character. Information theory is essentially a branch 
of mathematics. Although the language has a physical ring, the words 
information source, channel, coder, etc., are mathematical concepts physically 
inspired. The theory can be presented as a formal series of definitions, 
theorems, and comments. However, its relevance to a given problem is 
then not very clear. Section 2 provides contextual definitions and qualita- 
tive results: the later sections are more formal. 

2. GENERAL DEFINITIONS 

A Communications System 

Figure 1 shows the model of a communications system which is used in 
information theory. At the transmitter the source produces an output 
that is coded and fed into the channel. The channel output may be identi- 

Noise 



Source 






Coder 






Channel 






Decoder 






Sink 


, 


' 


, 


, > 


, 


. > 


i 


. > 





















Message 
Fig. 1. The model of a communications system which is used in information theory. 
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cal to the channel input, or it may be altered by noise or distortion. At 
the receiver the channel output is decoded and used. The model derives 
from communications, but it is applicable to the communications aspects 
of other problems. Examples. The storage of a digital computer is a 
channel, possibly noisy, with input and output separated by time. A 
control system may be a channel with electrical input and mechanical 
output. 
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The Information Source 

For purposes of the theory, the source in a given analysis is the point 
at which information enters the part of the system under consideration. 
The source may not actually generate information, but may merely store 
or relay it. Examples: a stack of telegrams waiting to be transmitted; a 
reel of recorded magnetic tape. Whether the source under consideration 
is a true generator of information or merely a storage point is not of concern. 

Controlled and Uncontrolled Sources. A controlled source is one 
which generates information at a rate controllable by the transmitter. 
Examples. The stack of telegrams being read by a telegraph operator is a 
controlled source; so is a speaker who may be slowed down by his audience 
when he speaks too rapidly for them to take notes. An uncontrolled source 
is one which produces information at a rate determined internally, which 
cannot be adjusted to the coding and transmission facilities available. 

Segmentation. The output of a source is a sequence of symbols. It 
is convenient to break this sequence into segments at a number of different 
levels. This process is called segmentation by the linguists. Example. The 
output of a teletype system is a sequence of binary selections, "Mark" 
and "Space." If these are denoted by "0" and "1," then and 1 are 
called the elements of the representation. 

The group of consecutive elements which represents a single letter, num- 
ber or mark is called a character or letter of the representation, and the set 
of all possible characters or letters is called the alphabet. The alphabet in 
a teletype system may be thirty-two characters in number, with each 
character a group of five elements. 

Words and Messages. In many alphabets one of the characters is 
called the space, and given special significance. Sequences of characters 
occurring between successive spaces are called words. A sequence of 
words which is more or less independent of the preceding and succeeding 
source output is called a message. Example. A single telegram might be 
called a message. If successive source symbols are highly correlated the 
whole (possibly infinite) source output is the message. 

Other Levels. The above list of levels is not exhaustive, nor are all 
these levels relevant to the description of a particular source. Examples. 
In written English the additional levels of syllable, phrase, clause, sentence, 
and paragraph are recognized, but the element level is not used. 

In the analysis of spoken language a set of elements, the distinctive 
features, have been introduced (Ref. 12). In a binary digital computer, 
element and character coincide as being the symbols and 1 used to rep- 
resent the binary digits. In a binary-coded-decimal machine the elements 
are the same, but the characters are groups of four, five, or six elements 
representing one decimal digit or one alphanumeric symbol. The group of 
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digits which fits into one storage register is called a word, and one line of 
coding, consisting of several related words, which is called an instruction 
in computer terminology, might be called a message. 

Choice of Levels. The choice of levels of segmentation is not stand- 
ardized outside linguistics, nor is there any agreement on terminology. 
When no particular type or level of segmentation is implied, the output 
of a source will be a sequence of symbols, selected from some finite alphabet. 
When two levels are needed at once, as in discussion of word-by-word 
translation of a sequence of letters, word will be used for the higher level 
and symbol or letter for the lower level. In mathematical discussion, a 
segment of a sequence will be called a message only if it is strictly statistically 
independent of preceding segments. 

Representations and Codes 

If a source makes a series of binary selections, its output may be rep- 
resented, for example, by a sequence of A's and B's or by a sequence of 
0's and l's. These are two representations of the source output. If the 
first is taken as primary, then the second is called a coded version of the first. 

Codebooks. To get from one representation to the other requires a 
dictionary, called an encoding codebook. This has two entries: 

A -> 

(1) 

to get back from the second representation to the first requires a decoding 
codebook with the entries 

-> A 

(2) 

Codes. A code is a transformation, which is defined by an encoding 
codebook or an equivalent set of rules. If a coded message is to be decoded, 
the inverse transformation as defined by the decoding codebook is also 
required. In the example of eqs. (1) and (2), the transformation is one- 
to-one on each symbol and defines its own inverse, so that only one code- 
book is required and it may be written with double-headed arrows: 

A <-» 

(3) 

BhI 

In representations with large alphabets the two codebooks may still be 
useful even if only one is necessary. Example. It is convenient to find 
a telephone number in a standard directory by looking up a name, but the 
inverse operation is tedious, though unambiguous. 
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Transliteration. A code is called a transliteration if each input 
symbol is transformed directly into one output symbol, so that symbol- 
by-symbol coding is possible and the number of entries in the codebooks 
is equal to the size of the alphabet. The code of eq. (3) has this property, 
but not all one-to-one codes do. Example. The representation of the 
binary source output as a sequence of A's and B's may be coded into a 
representation as a sequence of 0's and l's by the codebook 

AA -> 

AB -+ 10 

(4) 

BA -> 110 

BB -> 111 

To each input sequence corresponds one output sequence, which may be 
decoded into its original form by reversing the arrows in the codebook 
if the time origin of the coded version is known. The coded output is 
a one-to-one transformation on the input sequences, but it is not a trans- 
literation of the symbols A and B. Choose a different level of segmentation 
(word, rather than character) and let the first representation consist of 
sequences of the four words AA, AB, BA, BB. Then if the four words 
0, 10, 110, and 111 are taken as the dictionary for the second representation, 
the code becomes a transliteration. 

Significant Codes. In assigning code numbers to objects (Example: the 
items in a catalog) a distinction is made between significant and non- 
significant codes. Each code number may be considered as a coded version 
of a description of the item in English. A significant code is one in which 
transliteration is possible at some level of segmentation below that of the 
entire code number. Example. The code number assigned to a garment in 
a catalog may consist of a sequence of groups of decimal digits, the first 
group denoting type of garment, the second size, the third color. Each 
group may be independently decoded into English words, so that trans- 
literation is possible if each group is considered as a word in the coded 
version of the message. 

A code that cannot be decoded piece by piece is called nonsignificant. 
Example: a code assigning simple serial numbers to items in a catalog. 

Coding and Decoding Delay. Note that coding and decoding delays 
arise when the coding is not a transliteration. Example. In the codebook 
of eq. (4), after the source has selected its first symbol, the coder must 
wait until the second symbol has also been selected before it can encode 
the pair. In decoding, after a 1 has been received, it is necessary to wait 
for one or two more input symbols before the appropriate output pair 
can be selected from the set AB, BA, BB. 
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Representation and Selection. The output of a coder is called a 
representation of its input if it is obtained from the input by a one-to-one 
transformation with at most a finite encoding delay. Note that this 
definition agrees with the colloquial meaning of representation for a sig- 
nificant code, in which each segment of the output represents a corresponding 
segment of the input. 

A nonsignificant code requires a different interpretation. The code num- 
ber corresponding to a telegraphic greeting is not a modified version of the 
message, but an instruction as to where in the decoding codebook the 
message will be found. The coded version does not represent the message 
but selects it. This concept is basic for information theory (Ref. 13). 

Coders. The coder in Fig. 1 matches the source to the channel. The 
first requirement on the coder is that it match alphabets. It must transform 
sequences of symbols from the source alphabet into sequences of symbols 
in the alphabet which the channel will accept. This requirement does 
not specify the coder completely. The two codebooks of eqs. (3) and (4) 
both transform A's and B's into 0's and l's, but they describe different 
coders. 

Statistical Matching. For economy of transmission facilities the 
coder may be designed to minimize the number of channel symbols re- 
quired, on the average, per source symbol. This requires knowledge of the 
statistics of the source. Example. The codebook of eq. (4) is more compli- 
cated than the codebook of eq. (3) and introduces delay. However, if 
A's occur 99 per cent of the time and B's only 1 per cent, the output coded 
via eq. (4) will require only 0.5015 channel symbol per source symbol, 
whereas the output coded via eq. (3) will require one channel symbol per 
source symbol, and so will take nearly twice as long to transmit. Econo- 
mical coding will be discussed in Sect. 3. 

Channels 

A discrete channel, like a coder, accepts a sequence of symbols selected 
from its input alphabet and produces a related sequence of symbols selected 
from its output alphabet. The precise boundaries of the channel in a given 
system are a matter of choice. Example. A teletype system may be 
analyzed by using as a channel the medium of transmission of the electric 
pulses. A second analysis of the same system might treat the channel as 
running from input keyboard to output printer. 

As essential difference between a channel and a coder is that the channel 
output may not be an accurate representation of its input: some informa- 
tion about the message may get lost in transit. This may occur in two 
ways. 
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a. Loss. A channel is lossy if it is possible to make finer distinctions 
at its input than are preserved in its output. Example. Pulses of fixed 
duration and of any amplitude greater than may be used successfully to 
trigger a circuit which provides an output pulse of fixed duration and 
amplitude, no output being produced if the input pulse is smaller than 0. 
A channel is made of this circuit by using as an input alphabet pulses of the 
ten amplitude levels —5, —4, •••—!, +1, • • •, +5. This channel accepts 
ten input symbols and produces two output symbols : it loses the additional 
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Fig. 2. Some examples of discrete channels. 

amplitude information present in the input. Such a channel is shown 
schematically in Fig. 2a. 

b. Noise. A channel is noisy if a given input sequence may be received 
as any one of a number of possible output sequences, depending on random 
action of the channel. In a lossy channel the received sequence is de- 
termined by the transmitted sequence; in a noisy channel this is not true. 
Examples. Figure 2b shows a noisy channel in which the noise does not 
bother the receiver, who is still able to tell what has been transmitted, 
although the transmitter does not know exactly what has been received. 
Figure 2c shows a noisy channel in which the channel noise prevents either 
transmitter or receiver from knowing with certainty what happens at the 
other end of the channel. 

Decoders. Any of the channels of Fig. 2 can transmit information at a 
definite rate with an arbitrarily small probability of error. This can be 
done simply for the first two channels by merely lumping together some 
of the input symbols and output symbols, respectively. It can also be 
done for the channel of Fig. 2c, by making use of the proper coder and 



16-08 INFORMATION THEORY AND TRANSMISSION 

decoder. The coder is still like the codebooks of eqs. (3) and (4), although 
the matching job performed is more sophisticated. But the decoder is 
different. Since the channel performs a transformation on the input 
sequences which is many-to-many rather than one-to-one, the decoder 
must perform a many-to-one transformation. In the channel, each input 
sequence may produce many output sequences. The decoder must de- 
code all of these (or at least all of them which occur with appreciable 
probability) into the same output sequence, if it is to avoid making errors. 
This will be discussed in Sect. 7. 

3. SIMPLE DISCRETE SOURCES 

Self-Information Measure 

Let x denote a particular event, and let Prob {x} be its probability. 
The amount of information associated with the occurrence of the event x 
is defined to be 

(5) I(x) = -log Prob \x], 

where the choice of logarithmic base corresponds to the choice of a unit 
of information. The quantity I(x) is sometimes called self-information 
(Ref. 14) to distinguish it from the mutual information relating two events, 
discussed in Sect. 6. 

Units. Logarithms to the base 2 are chosen for eq. (5). The resulting 
unit is the bit, which is the amount of information associated with the 
occurrence of an event of a priori probability one-half. Other information 
units include the Hartley, which is the information given by an event of 
probability }{o, and the not, or natural unit, which is the information 
given by an event of probability 1/e, where e is the base of Napierian 
logarithms, e = 2.71828- 

Bits and Binits. In computer terminology bit is often used as a con- 
traction of binary digit. This practice cannot be followed in information 
theory, since the occurrence of a binary symbol with a priori probability 
other than 3^ does not provide a bit of information. The word binit 
will therefore be used as an abbreviation for binary digit (Ref. 15). 

Properties. The information measure has the following two important 
properties. 

1. Since Prob {x} =S 1 for any event x, 

(6) I(x) ^ 0. 

2. Let x and y be two statistically independent events, and let x, y 
denote the event which is their joint occurrence. Then 

(7) I(x, y) = I(x) + I(y), 
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since the probability of the event x, y is, by hypothesis of independence, 
the product of the probabilities of x and y. 

Distribution of Information 

Message Source. Consider a set M of n different messages, M = 
{mi}, 1 ^ i ^ n, and a random process that generates sequences by 
selecting messages from this set. (The word message implies that successive 
selections are statistically independent. See Sect. 2.) Let Xk be the kth. 
message selected in time sequence, — °o < k < w. Then Xk is a random 
variable, taking values from the set M = {mi}, with 

(8) Prob {x k = m^ = p(m t ) = p { 

as the probability of selecting the ith message as the kth. choice. As 
eq. (8) implies, it is assumed that the process is stationary, so that pi is 
independent not only of the earlier selections but also of the time index k. 
Bar Plot of Distribution of Information. The amount of informa- 
tion associated with the selection of message Mj is then also independent of 
k and of prior selections : it is given by 

(9) I (mi) = Ii = - log pi = - log p(mi). 

The random variable I(xk) takes its values from the set 

7(m») = I{, 
with probabilities 

(10) Prob {I{x k ) = U} = Pi. 

Because of eq. (9), if all the probabilities pi are different from one another, 
then on a bar plot of the distribution of information, the bars which give 
the probabilities of the different possible information values all terminate 
on the single exponential given by 

(11) Pi = 2" 7 *\ 

Mean and Variance. The information distribution is completely 
determined by the probabilities pi, via eq. (9). The most important param- 
eters of the distribution are its mean value, 

n n 

(12) I = - 2 Vdi = - Z) Vi log Pi, 

i=l i=l 

and its variance, 

(13) en 2 = (I - I) 2 = T 2 - I 2 . 
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Example 1. The source illustrated in Fig. 3 selects messages from the 
setilf = {A, B, C) with probabilities {0.755, 0.185, 0.060} and information 
values {0.405, 2.434, 4.059}. The information distribution has mean 
value 1.00 bit/symbol, and standard deviation o-/ = 1.10 bits/symbol. 
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Fig. 3. Bar plot of information distribution. 

The three message probabilities terminate on the exponential of eq. (11), 
shown dotted. 

Example 2. The source illustrated by Fig. 4 has an alphabet M = 
{0, 1}, with probabilities {3^, Yi\- This distribution also has a one-bit 
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Fig. 4. Bar plot of degenerate information distribution. 



mean value, but has zero variance, since the selection of either symbol 
requires the same amount of information. Note that in this case the total 
probability line occurring at / = 1 does not terminate on the exponential 
of eq. (9), because of the fact that the two symbols have the same informa- 
tion value. Such a distribution is called degenerate. The degeneracy 
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may be removed by shifting the probabilities slightly, to { 3/2 — 8, }/% + 5 } , 
so that the two bars wind up side by side rather than end on end. 

Rate and Entropy. For sources like those illustrated in Figs. 3 and 4, 
which generate sequences of messages rather than sequences of statis- 
tically related symbols, the average rate R at which the source generates 
information, in bits per source symbol, is just the mean of the information 
distribution. From eq. (12), this mean is a function of the probabilities 
{pi) of the message set only. When considered as a function of the proba- 
bilities, rather than as a mean of the information distribution, this quantity 
is called the entropy H of the probabilities {pi). Thus for these sources, 



(14) 



R = I = H bits/source symbol. 



For the more complicated sources to be discussed later the average rate 
R is still the average value of the self-information, but the averaging 



P(D 




0.25 0.5 

P(0)- 



Fig. 5. The entropy function H(p(0), p(l)) = -p(0) log p(0) - p(l) log p(l). 

operation is more complicated, and I no longer has the form of the entropy 
function of a probability distribution. 

The entropy function H{p\, p 2 ) is illustrated in Fig. 5. Since p\ -f- p 2 = 1, 
this is actually a function of a single variable only. 

Binary Coding 

The rate of a source is a significant parameter because it determines the 
communications facilities required to transmit the source output after 
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proper coding. The source in Fig. 4 generates information at a rate 
R = 1 bit/symbol, and each output symbol is just one binit. The curve 
of Fig. 5 shows that one bit is the maximum average amount of informa- 
tion that one binit can convey. In this case, the rate R has the interpreta- 
tion that the source output may be represented in binits so as to require 
R = 1 output binits per source symbol. This interpretation can be 
generalized to other sources. 

First Binary Coding Theorem {Controlled Sources). Given a discrete 
message source which generates information at an average rate R bits per 
message and given any 8 > 0, it is possible to construct a representation of 
sequences of messages as sequences of binary symbols so that, on the average, 
less than R -\- 5 output binary symbols are required per input symbol from the 
source. It is not possible to find a representation using fewer than R output 
binary symbols per source symbol (Ref. 16). 

A code which satisfies the requirements of this theorem does the job of 
statistical matching referred to in Sect. 2. 

Shannon-Fano Coding. The general strategy in constructing ef- 
ficient binary codes is to divide the message set into two subsets of nearly 
equal probability and to use the first digit of the coded output sequence 
to indicate in which half the selected message lies. Each half is divided 
into two subsets again by the next digit, and the process terminates on 
subsets which contain only one message (Ref. 17). 

This procedure is not quite explicit, however. It will not be possible 
to make all dichotomies equiprobable unless all the message probabilities 
are powers of Y^- If n °t, then there are many possible not-quite-perfect 
codes, and it is difficult to choose among them. The following procedure, 
called Huffman coding, is explicit, and it gives a "best possible" code 
(Ref. 18). 

Huffman Coding. 

1. List all possible messages in order of decreasing probability, and 
assign as the last digit in the coded output a to the next-to-last message 
and a 1 to the last message. These two messages will agree in all the 
(as yet unknown) digits preceding the last one. 

2. Merge the last two messages, adding their probabilities, and insert 
the sum in its proper position in the list of message probabilities. Now 
repeat step 1. Continue until all messages are merged. 

Example. The process is illustrated for the message set of Fig. 3 in 
Fig. 6, by a kind of a graph which is called a tree for obvious reasons. The 
code for each message is read off starting at the left node and reading the 
0's and l's which label the branches along the (unique) path terminating 
in the selected message. For Fig. 6 this leads to the codebook 
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A -> 



(15) 



p(A) = 0.755 

p(B) = 0.185 1.000 

p(C) = 0.060 




0.245 



Fig. G. A coding tree. 

Codewords. The Prefix Property. The output sequences of a code- 
book are called codewords. The code of eq. (15) and of Fig. 6 illustrates a 
characteristic feature of Huffman codes called the prefix property: no 
codeword is a prefix of any other longer codeword. 

The prefix property is a sufficient condition to guarantee that a sequence 
of codewords written down in order without spacing can be uniquely de- 
coded into a sequence of source symbols. Decodability is required in 
order that the output be a representation of the input, and spaces between 
words are not permitted since, if they were used, the output alphabet 
would be ternary and not binary. The prefix property is not necessary, 
however. It is possible to construct codes, which can be decoded after 
some delay, that do not satisfy this condition. 

Example. The codebook 

A -> 

(16) B -> 01 

C -> 11 

is decodable, but the codewords do not satisfy the prefix condition since 
is a prefix of 01. There is no advantage to such codes in the binary coding 
case, and there seems to be none in general (Refs. 19, 20, 21). 

The Szilard-Kraft Inequality. Let W{ be the number of binits in 
the codeword for the ith message m t -. Thus for the codebook of eq. (15) 
one has 

w 1 = 1 

(17) w 2 = 2 

w 3 = 2. 



16-14 INFORMATION THEORY AND TRANSMISSION 

The smaller the W{ are, the fewer output binits are required per input 
symbol. However, if they are too small, the codewords cannot all be 
different. Thus if w { = 1 for all i, the only distinct codewords are and 
1, which cannot distinguish three different messages. A condition on the 
lengths of code words is given by: 

The Szilard-Kraft Inequality. Given a set of n messages, it is possible 
to assign a codeword of length W{ to the i-th message, and to satisfy the prefix 
condition, if and only if the W{ satisfy the inequality 

n 

(18) ]£ 2~ w * ^ 1. 

If the codeword lengths do not satisfy this condition, no decodable code 
can be constructed (Refs. 22, 23). 

Coding Implications. Suppose all pi are powers of 3^, so that all 
information values are integers. Then let W{ = I{ — — log p{. This gives 

n n n 

(19) E 2~ w < = E 2 log Vi = £ p { = 1, 

which satisfies the constraint of eq. (17). Thus a decodable code can be 
constructed in which each message has a codeword length in binits equal to 
its information content in bits. Then the average codeword length, which 
is the average number of output binits per input message, is 

n n 

(20) w = X PfWi = Zl Vih = I = R. 

i=l i=l 

It can be shown that no smaller value of w can be obtained from code- 
word lengths satisfying eq. (18). This proves the First Binary Coding 
Theorem for these special cases, with 8 in the theorem = 0. 

Example. Consider the following set of five messages and their codes. 

Message Probability Codeword I{ Wi 

mi \ 

m 2 I 

m s | 

m 4 He 



m 5 



u 






1 


1 


10 


2 


2 


110 


3 


3 


1110 


4 


4 


1111 


5 


5 



General Case. In general the I{ are not integers, since the pi are not 
powers of 3^- However, a decodable code can always be constructed in 
which Wi is the smallest integer which is greater than or equal to /;. Then 



(21) 
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Ii ^ Wi < I{ + 1, 
Vdi ^ Pi">i < Vili + Pit 
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so that the average number of output binits per message is never more 
than one in excess of the average number of bits per message. This means 
that if the number of bits per message is large, the percentage excess is 
small. One can always make the number of bits per message large by 
coding sequences of input messages, taking all possible sequences of length 
L messages as a new message set, containing n L different messages. 

Example. For the codebook of eq. (15) and the source of Fig. 6, the 
codeword lengths W{ are given in eq. (17). One can compute w, the average 
number of binits per source symbol: 



(22) 0= Ep, 



Wi 



i=i 



= 1 X 0.755 + 2 X 0.185 + 2 X 0.060 = 1.245 binits/bit. 

Now form all 3 2 = 9 possible pairs of messages selected by the source 
of Fig. 6. Using the Huffman coding procedure, as illustrated by the tree 



1.0000 




--AB 
0.1397 



Fig. 7. A coding tree for message pairs, 
in Fig. 7, gives the following set of messages, probabilities and codes. 



Message 


Probability 


Code 


AA 


0.5700 





AB 


0.1397 


11 


AC 


0.1397 


101 


BA 


0.0453 


1001 


BB 


0.0453 


10001 


BC 


0.0342 


100000 


CA 


0.0111 


1000011 


CB 


0.0111 


10000100 


CC 


0.0036 


10000101 
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Evaluating eq. (22) in this case gives w = 2.0767. The average informa- 
tion per message is two bits: for since successive messages are statistically 
independent, the information in a sequence of two messages is the sum of 
the informations in each of the two, and the average of a sum is the sum 
of the averages. The efficiency of this coding in bits per binit is about 
0.96. In terms of the inequality of eq. (21) this might be as low as % or 
as high as 1.00. The fact that the coding here has an efficiency well 
above its lower bound is typical of coding results. It is due to the fact 
that the entropy curve in Fig. 5 has a very broad maximum, so that the 
message set must be quite skewed in probabilities before the efficiency of 
coding drops very low. 

General Case Continued. Define wl as the average number of binits 
required to code a block of L source symbols. Rewriting inequality (21) 
for the new message set gives 

LR ^w L ^ LR + 1, 

(23) 

wl R + 1 

R ^ — ^ 

L L 

Now wz,/L is the average number of output binits per original input 
message, so that the last line of eq. (23) satisfies the requirements of the 
First Binary Coding Theorem for any L > 1/5. This proves the theorem 
in the general case of a message source with arbitrary information distribu- 
tion. It also justifies a definition which assigns the same rate to the two 
very different sources of Figs. 3 and 4, if these sources can be controlled. 

Controlled Source Coding. The source of Fig. 4 may be controlled to 
read out one binit per second. The source of Fig. 3 may be controlled so 
that its coded output produces one binit per second. This will require that 
the source be speeded up when it generates A's, and slowed down when 
it generates B's or C's, but on the average it will be generating very nearly 
a symbol per second. The average rate of the source, then, determines 
the communications facilities required to transmit its encoded output. 
The differences in the distributions of Figs. 3 and 4 affect only the amount 
of delay and the size of the codebook required for efficient coding into 
binits. This result holds for any controlled source, whose symbol rate 
may be varied in order to keep its information rate constant. 

Uncontrolled Sources. Here the source generates messages at a 
constant rate. If there is any variance in its information distribution, the 
rate at which it generates information will then fluctuate. In an efficient 
code, Wi, the number of output binits in the codeword for the message 
mi, is still close to the message self-information I (mi). The average number 
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of binits per message wl is still near to the source rate R. But it is possible 
(though highly improbable) that all L of the messages in a block will 
be those of maximum self-information. Therefore it is not possible to 
transmit all message sequences as they come along unless a channel is 
used which can transmit binits at a rate equal to itf max times the (fixed) 
rate at which the uncontrolled source generates messages. Here w max is 
the largest of the W{, i.e., the length of the longest codeword. 

Minimax Coding. If all the message sequences generated by an un- 
controlled source are to be transmitted unambiguously, the best code to 
use is not the Huffman code, which minimizes w but will have a large 
w max if the information distribution has appreciable variance. Rather it is 
better to use a code that minimizes the value of w milK , a minimax problem 
with a simple solution (Ref. 24). The codewords are taken of uniform 
length w m , where w m is the integer such that 

(24) 2 Wm ~ l <n m ^ 2 Wm , 

or 

w m - 1 < log 2 n m ^ w m . 

Here n m is the number of messages. Since there are 2 Wm codewords of 
length w m , there are at least enough to label all the messages. Note that 
this coding procedure is quite independent of the message probability or 
information distribution and depends only on the number of messages in 
the set. 

Efficient Coding : Uncontrolled Source. The average rate R at which 
a source generates information still has significance when the source is 
uncontrolled. Not all sequences of L messages can be coded into about 
LR binits, but almost all of them can. More precisely we have: 

Second Binary Coding Theorem (Uncontrolled Sources). Given a 
discrete message source which generates information at an average rate R 
bits per message, and given any 5 > and any e > 0, it is possible to construct 
a representation of sequences of messages as sequences of binary symbols so 
that, for each message sequence, less than R -\- 8 output binary symbols 
are required per input symbol from the source, except for a set of message 
sequences whose total probability is less than e. 

The procedure is to code the messages in blocks of length L, coding 
each block into a codeword of length W{ ^ I{ + 1. The theorem follows 
because the self-information of a sequence of messages is the sum of the 
self-informations of the component messages (since statistical independence 
is assumed), and because the sum of a large number L of identically distri- 
buted, statistically independent random variables is very likely to be very 
near, percentagewise, to L times the mean of the distribution. 
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Sums of Random Variables. The Second Binary Coding Theorem 
follows from the weak law of large numbers. Stronger results derive from 
the Tchebysheff inequality and the central limit theorem. These three 
results applied to self -information are: 

1. Weak Law of Large Numbers. For any e > and any 8 > 0, 
an integer L Q can be found so large that the probability that a sequence ofL > L 
messages will have an amount of self-information greater than L(I + 8) is < e. 

2. Tchebysheff Inequality. For any 8 > 0, the probability that a 
sequence of L messages has an amount of self-information greater than L(I + 8) 
is < e = o- 7 2 /L5 2 . 

3. Central Limit Theorem. For any 8 > 0, the probability that a 
sequence of L messages has an amount of self-information greater than 
L[I + {8/s/Li)] is asymptotically given by the expression 

l r x l r x 

(25) e = , e-zWrS dx = I e -i/»/2«r J * dy 

V2irL(T I 2 J x =8iVZ V 2tto-/ J y=s 

Here I is the mean and aj 2 is the variance of the self-information distribu- 
tion of the messages. 

Coding Interpretations. Each of these results translates into a result 
for efficient coding, since by eq. (21) it is possible to assign distinct binary 
codewords to all possible message sequences of length L so that the differ- 
ence between codeword length in binits and information in bits is less than 
unity for each sequence. Thus to every sequence in a set of sequences of 
total probability ^ 1 — e, we can certainly assign codes of length 
< L(I + 5) + 1. The remaining sequences, of total probability ^ e, may 
all be assigned the same codeword, and will cause ambiguity or error a 
fraction e of the time. 

Storage and Delay. The central limit theorem shows that for fixed 
error probability e, the difference between wl/L, the binits per message, 
and I, the bits per message, decreases with blocklength L only like l/-\/L. 
This implies that it may be necessary to use much longer blocks to get 
efficient coding for an uncontrolled source than would be required for a 
controlled source, for which the difference between wl/L and I = R de- 
creases like 1/L, as in eq. (23). 

Effective Number of Messages. From the weak law of large numbers, 
there is a set of message sequences of total probability > 1 — e, each 
sequence of which has self-information within ±L5 of LI. Each sequence 
in this set then has probability in the range 

(26) 2 L(f±a) , 

and the total number of sequences in the probable set, for large L, lies 
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in the range 

(27) (1 ± e) 2 L(7±a) , 

no matter how small e and 5. Adding unity to L ultimately multiplies the 
number of messages in the probable set by 2 1 . This is what would happen 
if there were just 

(28) n„ f £ = 2' 

different messages in the set, and n e ff is therefore called the effective num- 
ber of messages, or the effective alphabet size. Since I ^ log n, we have 

(29) neff ^ n. 

That is, the fact that all messages are not equiprobable produces a growth 
in the probable sequence set as if a smaller equiprobable message set were 
being used. 

4. MORE COMPLICATED DISCRETE SOURCES 

Most natural sources are more complicated than those discussed in 
Sect. 3. A more general source is a random process which generates 
sequences of symbols like the letters of English text, in which each symbol 
is selected with a probability which depends on the values of the preceding 
symbols. 

Joint Probabilities. Consider a set S of n different symbols, 

S = {s { }, l^i^n. 

Let x/t be the symbol selected at (integer) time k, — ■» < k < +oo. Then 
Xk is a random variable, taking values from the set S = { s» } . The random 
process is well defined if the joint probabilities 

Pi(zfc) 

P2(%k, Xk-i) 

(30) 



Pj+l (X k , Xk-l, •", Xk-j), 

are known for all combinations of x values and all values of j. It will 
be assumed that the process is stationary (and indeed ergodic), so that the 
probabilities are independent of the time index k. 
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Conditional Probabilities. Knowledge of the joint probabilities of 
eq. (30) is equivalent to knowledge of the conditional probabilities 

3o(s») 

qi(si\x k -i) 

(31) ; 

Qj\ s i\ x k—1, x k—2) ' ' ', x k—j)- 

The two sets of probabilities of eqs. (30) and (31) are related by 

qo(si) = pi(s t ) 

(32) qj(si\xk-i, Xk-2, '••, Xk-j)Pj(xk-i, x k - 2 , ■ • •, x k -j) 

= Py+i(st, Xk-i, x k -2, •••, Xk-j). 
Markov Sources 

If for some integer N and all integers j > 

(33) q N+J -(Si\xk-i, Xk-2, • • -, Xk-N-j) = qN(si\x k -i, x k -2, • ■ •, xji-n), 

so that a knowledge of N preceding symbols gives all the probabilistic 
information available about the next symbol value, then the process is 
a multiple Markov process of order N (Ref. 52). When iV = 1, the process 
is called a simple Markov process. 

Self-Information. If the process is Markov of order N, then the 
self-information provided when the symbol s; occurs is the negative log- 
arithm of its probability, but this is now a conditional probability depend- 
ing on the values of the preceding N symbols. The self-information is 
thus a random function of the N random variables Xk—i, Xk—2, • • ' , Xk-N- 

(34) lN(si\x k -i, Xk-2, ••-, Xk-N) = — log qxisilxk-i, x k - 2 , • • •, X k -N)- 

Average Self-Information. A self-information of the symbol S{ which 
is not a random function may be obtained by averaging eq. (34) over the 
conditional probability r N {xk—\, Xk—2, • • ' , £&— vK) that when s t - occurs 
at time k, the preceding N symbols will have the values Xk—\, Xk—2, — • , 
Xk—N- By Bayes's theorem, 

PN+l(Si, Xk—l, Xk—2, ' • ', Xk—x) 
Pl(Sf) 

r N (x k -i, X k -2, "•, X k -N\Si) 

X log giv(s,- 1 3jfc_i, X k -2, •", X k -N) 



(35) 


TN{Xk-\, X k -2, '", X k -N\Si) = 


and 




(36) 


In(s{) = - X) ••• Z) 5 
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is defined as the average self-information of the symbol s t - for an iVth order 
Markov process. 

Source Rate. The average rate R at which a Markov source generates 
information is equal to the average In of the In(Si) over the probabilities 
of the symbols pi(si). This gives the average self-information per symbol 
of the process: 

n 

(37) R = I N = S Pi(8i)I N (si). 

Other Sources 

If the source is not a Markov process of finite order, the self-information 
of a symbol may not be well defined, since it may then be a function of an 
infinite number of random variables. However, the quantity 7jv(si|£fc_i, 
Xfc_ 2 , • • •, xjc-n) given by eq. (34) is still defined for each N, and it is 
called the iVth order conditional self-information of s,: the quantity 
In (si) defined by eq. (36) is called the N-th order average self-information 
of the symbol S{. It can be shown that for any process, for all s* and any 

Iif(8i) ^ 
(38) 

In (si) ^ 7jv+i(s»); 

the average information provided by the occurrence of S{ when the N 
preceding symbols are known is a monotone decreasing function of 
N. Further knowledge of the past, on the average, makes whatever symbol 
happens next more probable, and therefore less informative. 

Upper Bounds on Source Rate. If an information source has a 
measured first-order probability distribution pi(si), the average self-in- 
formation of each of its symbols is at most equal to — log pi(si), the value 
it would have if successive symbols were statistically independent. If 
the source has a given conditional distribution of order N, the average 
self-information of each symbol is bounded above by the average self- 
information In (si) of a Markov process of order N with the same iVth 
order conditional distribution. Again, statistical dependence beyond what 
is contained in the given distribution can only reduce the average self- 
information of each symbol. 

Self-Information of a Symbol. From eq. (38) it follows that one can 
define a limit, 

(39) I(Si) = lim I N (Si), 

which converges for each s z - to a non-negative number. The limit, I (si), 
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is the average self-information added by the occurrence of the symbol S{ 
when all the preceding symbols are known. 

Note that no general lower bound better than can be obtained. A 
process which looks random on the basis of Nth. order statistics may always 
be deterministic on the basis of statistics of order N + 1 and have an 
average rate of zero. 

Source Rate. The average self-information I of the process is again 
equal to its rate R, and is given by either of the two expressions 



(40) 



1= HPl(Si)I(Si) 

siCs 



= lim In, 

N -► 00 



where In, the average Nth order self-information of the process, is given 
by eq. (37). 

More General Sources. The only type of source of greater generality 
than the multiple Markov process of finite order which has been studied 
in detail is called a finite-state source (Ref. 25). Note. A finite state 
source (a) includes the Markov processes of finite order but it is not in- 
cluded among them and (b) is still not general enough to generate all and 
only the grammatical sentences in English word by word (Refs. 26, 27). 
Because of the complexity of natural sources like written language, indirect 
methods must be used to estimate their rate. A straightforward applica- 
tion of the definitions involves the measurement of probability distributions 
of order so high that in all the written English there would be too small a 
sample for an accurate estimate. 

Coding and Delay 

The first binary coding theorem still applies to more complicated sources. 
It carries over unaltered to any discrete ergodic source, whether it be 
multiple Markov, finite state, or still more general. This may be shown 
by two coding methods. 

1. Block Coding. Segment the source output into blocks of length 
L, and code each block from a fixed codebook containing a binary sequence 
for each of the n L possible sequences of source symbols. Each source 
sequence may be coded as before into a number of binits at most one 
greater than its total information content in bits. 

The average self-information in a block of L source symbols is equal to 
J , the average self-information of the first symbol in the block when no 
past history is known, plus 7i, the average self-information of the second 
symbol when the first is known, etc., the last term being Il—\, the average 
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self-information of the Lth symbol when all preceding L — 1 are known. 
Averaging over all sequences gives wl, the average number of binits per 
block of L input symbols, as bounded by 

L-\ L—\ 

(41) £ Ij < w L S E Ij + 1. 

Dividing eq. (41) by L gives the average number of binits per source 
symbol : 

1 L - 1 _ w L 1 L - 1 _ 1 

(42) 7 E^^ T ^ 7 Z^ + 7- 

Li y=o Li Li j = o L 

The summation in eq. (42) is the average of the first L average self-informa- 
tions of the process. Since Ij cannot increase with j, the average will in 
general be greater than the limit I = R, but it will approach the limit 
as L — > co. Given any 5 > 0, it is always possible to find an L so large 
that the first coding theorem is satisfied, but the required L may be very 
large even if the process is Markov of small order. Example. A simple 
Markov process (of order one) has I = 10 bits per symbol, and I\ = 
R = Mo bit per symbol. It will take L = 100 to code the output of this 
source at 50 per cent efficiency, i.e., at two output binits per input bit. 

2. Conditional Coding. A more complicated but more efficient 
procedure is conditional coding. Here blocks of L 2 symbols are encoded 
by using one of n Ll codebooks. The codebook to be used is determined 
by the preceding L\ symbols. Such a process has an encoding delay of 
L = L\ + L 2 input symbols, and an average number of binits per source 
symbol given by 

1 L ~ x _ w L2 1 L ~ x _ 1 

(43) - E h ^ ~T ^ T 2 7 ' + 7" 

Li2j=Li L12 Li2j=Li L/2 

Example. In the simple Markov example given for block coding, 
conditional coding will give better than 50 per cent efficiency for L\ = 
1, L 2 = 10, L = 11, which is much less delay and a much smaller code- 
book than is required by the L = 100 above for block coding. 

Correlated Information Values. In extending the second binary 
coding therorem to more complicated uncontrolled sources, an additional 
kind of delay arises. The occurrence of a symbol with self-information 
value above the mean may make it more probable that the succeeding 
symbol will also have self-information above the mean. Then it will be 
necessary to encode much larger blocks of symbols in order to make it 
highly probable that the percentage deviation of the self -information of the 
block from its mean value will be small. Mathematically the problem 
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becomes one of adding correlated rather than statistically independent 
random variables, and convergence to the mean may be much slower. 
The second binary coding theorem itself extends to a very broad class of 
sources, but the Tchebysheff inequality and the central limit theorem do 
not apply in the form given above. 

5. DISCRETE NOISELESS CHANNELS 

Type I. Channels. The simplest noiseless channel is one in which each of 
the set S = { s» } of symbols which may be applied as an input to the channel 
is received unaltered at its output. Since there is a one-to-one correspond- 
ence between input and output symbols, they may be identified and 
called channel symbols. If all channel symbols are of equal duration, 
their number n and their common duration t completely specify the channel. 
Such a channel will be called a channel of type I. 

Channel Capacity. The capacity of a noiseless channel is the maximum 
average rate at which information can be received over it. The capacity 
may be measured in bits per symbol, denoted by C, or in bits per second, 
denoted by C t . If the common symbol duration is t seconds, then 

(44) tC t = C. 

For a type I channel, inequality (38) and the following discussion show 
that statistical dependence between successive symbols cannot increase 
the rate of transmission. Therefore the capacity can be computed by 
assuming statistical independence and maximizing the average rate R 
in bits per symbol. 

(45) R = - 2 Pi(s.O log Pi (s;) 

si£S 

with respect to variations in the probability distribution pi(si). But this 
rate is just the entropy of the Pi(s{) distribution, which has the maximum 
value log n, attained when all p\ (s») are equal to 1/n. Thus for a type 
I channel, 

C = log n bits per symbol, 
(46) 

C t = (1/0 log n bits per second. 

Redundancy. If a source is connected to a type I channel and selects 
channel symbols with unequal probabilities or with statistical dependence, 
the rate R = I at which it generates information will be less than the 
capacity C of the channel. The difference C — R is defined as the (absolute) 
redundancy, in bits per symbol, of the source with respect to the channel. 
The ratio of absolute redundancy to channel capacity, a number between 
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and 1, is defined as the relative redundancy of the source with respect 
to the channel. In terms of the interpretation at the end of Sect. 3 of 
R = I as the logarithm of the effective alphabet size of the source, the 
redundancy is a measure of the reduction in logarithm of the size of the 
effective alphabet, due to nonoptimum utilization of the channel. 

Type II Channels. A more complicated noiseless channel, which will be 
called type II, has a different duration U for each channel symbol S{. 
It is again true that capacity is attained by using symbols chosen with 
statistical independence and maximizing the average rate R t (now in bits 
per second) with respect to the symbol probabilities pi(s{). But the rate 
is now given by 

- £ Pi(s») logpi(sf) 

Si £S 

(47) R t = . 

2^ Pi(*iX- 

Si£S 

and the maximization leads to the condition that the instantaneous rates 
at which each symbol transmits information all be equal to the channel 
capacity C. Thus 

(48) C t = ^- = -(-)log Pl (*), 

lr \vi/ 



or 



Vl ( Si ) = 2- c <\ 



The capacity C t is determined, for given durations U, by the normaliza- 
tion requirement that the probabilities of the symbols sum to one; this 
gives 

(49) £ Pi(*<) = S 2-°A = 1, 

and Ct is the (unique) real root of this equation. Redundancy with respect 
to a channel of type II is defined as it was for a channel of type I, using 
R t and C t rather than R and C. Notice that a source is redundant with 
respect to a type II channel unless the probabilities with which it chooses 
symbols are unequal, and are given by eq. (48). 

Type III Channels. Shannon (Ref. 28) discusses a finite-state channel, 
which will be called type III. Here the symbols are of different durations, 
and the alphabet of symbols available at each instant depends on the 
preceding symbols which have been sent over the channel. An expression 
for the capacity of such a channel has been given (op. cit.). Type III 
channels have storage. They will not be discussed here, except to note 
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that for each such channel there is a corresponding finite-state source, 
which has no redundancy with respect to the given channel. This optimiz- 
ing source no longer selects successive symbols with statistical independ- 
ence. 

Noiseless Channel Coding Theorems. The capacity C of a noiseless 
channel is a rate at which some particular optimizing source can transmit 
information over the channel. Since the source whose output may need 
transmission will not usually be an optimal source, this is not a justification 
for considering V to be an important channel parameter. The justification 
is that given a channel of capacity C, any source of rate R < C, and no 
source of rate R > C may be so encoded as to permit reliable transmission 
over the channel. 

Binary coding theorems one and two can be interpreted as showing how 
any source can be coded into a binary noiseless channel of type I. These 
theorems can be generalized. 

Noiseless Channel Coding Theorems 

I. Controlled Source. Given a discrete controlled source and a discrete 
noiseless channel which has capacity C t bits per second, it is possible to control 
the source to any average rate R t < C t and to encode its output for unambiguous 
reception over the channel. This is not possible for any R t > C t . 

II. Uncontrolled Source. Given: a discrete uncontrolled source of type I, 
II, or III with average rate R t bits per second; a discrete noiseless channel of 
capacity C t bits per second; and any 5 > 0. If R t < C t , it is possible to 
encode sequences of source symbols for transmission over the channel so that 
the probability that such a sequence will be incorrectly decoded is < 8. This 
is not possible if R t > C t . 

6. DISCRETE NOISY CHANNELS. I. DISTRIBUTION OF INFORMATION 

Mutual Information 

Let x and y denote two related events, and let x, y denote the event 
which is their joint occurrence. Let Prob {x}, Prob {y}, and Prob {x, y\ 
be the associated probabilities. 

The self -information given by the occurrence of x is defined in eq. (5) as 

(50) I(x) = -log Prob {x}. 

If y is now observed, and x and y are not statistically independent, the 
probability of x a priori will be changed a posteriori to 

Prob \x,y\ 

(51) Prob \x\y\ 



Prob {y\ 
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This change in the probability of x changes the amount of information 
required to select it to 

Prob {x, y\ 

(52) I{x\y) = - log Prob \x\y\ = - log • 

Prob \y\ 

The difference between eqs. (50) and (52) measures how the amount of 
information required to select x is changed by the knowledge of y. This 
difference is denoted by I(x; y), the amount of mutual information between 
x and y. Then 

(53) I(x; y) = — log Prob {x} + log (Prob [x, ?/}/Prob {y}) 

= — log Prob {y} 4- log (Prob {x, ?/}/Prob {x}) 

Prob {x, y) 

Prob {x\ Prob {y\ 

Mutual information is measured in the same units as self-information 
(see Sect. 3). 
Properties. 

1. I(x; y) is symmetric: '■ 

(54) I(x; y) = I(y; x). 

This follows from the last line of eq. (53), and justifies the name "mutual 
information." 

2. I(x; y) vanishes if x and y are statistically independent. If not, 
there is a decomposition generalizing eq. (7) : 

(55) I(x, y) = - log Prob {x, y) = I(x) + I(y) - I(x; y), 

showing that I(x; y) plays the role of a correlation (Ref. 2). If Prob 
[x, y) is greater than Prob \x) Prob \y) then I(x; y) is positive. 

3. I(x; y) may be positive or negative, but cannot be greater than the 

self-information of x or y : 

I(x;y) £I(x) 
(56) 

SKy). 

This follows from eq. (53), since the conditional probabilities are at most 
unity, and have nonpositive logarithms (Refs. 14, 29). 

Notation. Any I function whose argument contains no semicolons is 
interpreted as the negative logarithm of the probability of its argument: 
thus I (x | y) = — log Prob {# | y}. Any 7 function whose argument contains 
a semicolon between two sets of variables is interpreted as in eq. (53), 
where x and y stand for the expressions to the left and right of the semi- 
colon, and x, y stands for their conjunction. 
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Distribution of Mutual Information 

Mutual information measures how much information one symbol pro- 
vides about another. It can be used in the discussion of Sect. 5 on sources 
which generate sequences of related symbols. Here it will be applied only 
to the discussion of noisy channels. 

Discrete Noisy Channel. Consider a simple noisy channel, as il- 
lustrated in Fig. 2. There is a set U = {u{}, 1 ^ i ^ n u , of symbols 
which may be transmitted, and a set V = {v 3 -}, 1 ^ j :g n v , of symbols 
which may be received. It will be assumed throughout that the channel 
is without storage and that it and the source are stationary: the probability 
of transmitting the symbol U{ and receiving the symbol Vj is independent 
of time and of prior transmissions and receptions. Let x k and y k be the 
transmitted and received symbols at (integer) time k, x k G U and y k G V. 
Then the pair (x k , yk) is a stationary random variable, taking values from 
the set U X V = [u{, Vj} of ordered pairs of transmitted and received 
symbols, with probabilities 

(57) Prob [x k = u i} y k = Vj} = p(u i} Vj). 
Denote the first order probabilities by 

n v n u 

(58) p(ui) = 2 P(Vi, vj), q(vj) = 2 P(u it vj) 

3=1 j~l 

and the conditionals by 

p(U{, Vj) p(U{, Vj) 

(59) pMvj) = — — — . q(vj\Ui) = — — 

q(vj) p(ui) 

Mutual Information of a Noisy Channel. The amount of informa- 
tion given by y k about x k is also a stationary random variable, which takes 
values equal to the numbers I(ui) Vj) with probabilities p{ui, Vj). 

The distribution of this random variable is completely determined by 
p(u{, Vj), through eqs. (53) and (58). Its most important parameter is its 
mean value, the average rate R at which the received symbols give informa- 
tion about the transmitted symbols: 



(60) 



#=22 P( u i> v i)I(Ui; Vj), 

^ ^ p(u if Vj) 



i=l j=l 



p(Ui)q(vj) 
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Although for particular m, v } - the mutual information may be negative, 
the average R of eq. (60) is always positive. 

Example 1. The Binary Erasure Channel. As illustrated in Fig. 8, this 
channel accepts two input symbols, and 1, and produces three output 
symbols, 0, 1, and X. With probability p its output reproduces its input; 



P (0) = i o = m « *. a9 > » vi = 

^V^ n i 




V 2 = X 



| 1.0- 

' 0.8- 

| 0.6 

E 0.4- 
o 

& 0.2- 



-1.0 



p(l)=I 1 =.Ui* r Q9 > .173=1 

Information, bits 
Fig. 8. Binary erasure channel and mutual information distribution. 

with probability q, its input is erased and an output X indicates the erasure ; 
and 1 are transmitted with equal probability. 

When a or a 1 is received, the mutual information is 



(61) 



Prob {ui, Vi} Prob {yxlwx} 



rroDiui, t>ij rroo [v t \u 

7(0; 0) = log — = 

Prob {wi} Prob {vi} Prob {^i} 



= log 2 = lbit = 7(1; 1). 



_ log? 

~~p/2 

When an X is received, 

Prob {y 2 |wi} q 

(62) 7(0; X) = log \ ' ' = log - = log 1 = 

Prob [Vi] q 



= 7(1;Z). 
information distribution shown in Fig. 8, with the average 



This gives the 
value 

2 3 

(63) R = Yj 2 P( u i> v i)I(ui'> Vj) 

1=1 j=l 

= (p/2)/(0; 0) + (g/2)7(0; X) + (p/2)/(l; 1) + (g/2)7(l; X) 
= 2(p/2) + 2(g/2) X = p. 
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Example 2. The Binary Symmetric Channel. As illustrated in Fig. 9, 
this channel also accepts the two input symbols and 1, but it only produces 
the same two output symbols. With probability p its output reproduces 

1.0 
p(0) = | = u l K w * > / O^Vi j.0.8 

I 0.6 
.2 0.4 
I 0.2 
p(l) = i l = u 2 S R/h ,\ l = v 2 L I-L 




8 ^ -3.0 -2.0 -1.0 0.0 1.0 

Information, bits — >- 

Fig. 9. Binary symmetric channel and information distribution. 

its input: with probability q = 1 — p its output is the incorrect symbol. 
and 1 are transmitted with equal probability, and q is < 3^- 
When the correct symbol is received, the mutual information is 

Prob {vilwi} 

(64) 7(0; 0) = log » ' ' = log p/(h) = log 2p > 

Prob [vi] 

= /(l;l). 
When an error is made, 

Prob {y^lw*} 

(65) 7(0; 1) = log ' ' = log q/(h) = log 2q < 0. 

Prob \v 2 \ 

This gives the mutual information distribution illustrated for q = ^ 
in Fig. 8, with the average rate 

(66) R = p log 2p + q log 2q — p + q -\- p log p + q log g 

= 1 - ff (p, ff ), 

where H(p, q) is the entropy function illustrated in Fig. 5. For q = X A, 

H(p, q) = H(f , i) = 0.5032, 
and 

R = 0.4968 bit per symbol. 

Averages of Information Measures. In addition to the average rate 
R, other averages of information measures must be considered. 

Notation. An average of an information function I{ui) Vj) over the 
joint distribution p(u{, Vj) is denoted by replacing the names "mj" and 
11 Vj" of the symbols by the names "U" and "V" of the sets from which 
the symbols are selected. A single capital denotes an average over a 
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univariate distribution. Thus 

n v n u 

I(U; V) = R = X) Z) P( u i, Vj)I(ui; v,) 



I(U) =H({p( Ui )}) = EpWW 



(67) 



»=i 



i=i y=i 

n u n v 

= -£L p(Wf, »i) logp(i*,-|«y). 
z=l j=l 

Equivocation. The average rate at which information about the trans- 
mitted symbols is supplied to the channel is the average self-information of 
the transmitted symbols. This by eq. (56) is greater than the average 
rate at which such information is received. The difference is 

I(U) - I(U; V) = - 2^ P(m) log p(Mi) - Z_, 1^ V( u i> vj) log — — — - 
i=i z=i y=i p{Ui) 



n„ n„ 



= -LE V(ui, i>j) log p(Ui) 
i=i y=i 

(68) ^ ^ p(ti,-|»y) 

= - .L .L p( m * u i) log — 7-7- 
»=i y=i PW 

= -EE P( M »'» y i) logp(wiky) 
;=i y=i 

= Z(C7| TO ^ 0. 

This quantity is the conditional entropy of the set U given 7. It measures 
the average amount of information about the transmitted symbol which 
the receiver still lacks after noisy reception, and thus the average rate at 
which it would be necessary to transmit additional information over an 
extra channel in order to make the receiver certain of each transmitted 
symbol. This quantity is also called the average equivocation of the 
received symbols. Equivocation is present in the channels of Figs. 2a 
and 2c, but not in Fig. 2b. 

Irrelevance. The average rate at which the received symbols give 
information (subject matter unspecified) is the average self-information 
of the received symbols. From eq. (56), this is also greater than the 
average rate at which information is received about the transmitted symbols. 
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The difference may be shown, as in eq. (67), to be 

I(V) - I(U; 7) = -EE Viui, vj) log q(vj\ui) 
(69) 

= i(7\u) ^ o. 

This quantity is the conditional entropy of V given U. It measures the 
amount of received information not relevant to the transmitted information, 
but relevant only to the channel noise. 

The names "Spread," "Dispersion," and "Prevarication" have all been 
used for this quantity. "Irrelevance" seems more appropriate in the case, 
for example, of the channel of Fig. 26, in which the receiver receives in- 
formation which is irrelevant but not misleading. The channel of Fig. 2a 
has no irrelevance, but that of Fig. 2c does. 

7. DISCRETE NOISY CHANNELS. II. CHANNEL CAPACITY AND 
INTERPRETATIONS 

The Noisy Channel Specification 

Formally a noisy channel without storage is a graph, like those in Figs. 
2, 8, and 9, in which each branch connects a transmitted symbol with a 
received symbol and has a number on it. The numbers are the conditional 
probabilities q(vj \ Ui) which define statistically what the channel does to 
given input symbols. These numbers are fixed for a given channel, but the 
transmitter is free to decide how to use the input symbols. Only the channel 
with no storage will be considered. 

Transmitter Strategy. The transmitter strategy formally is a random 
process, selected by the transmitter to generate sequences of transmitter 
symbols for transmission over the noisy channel. If an input message 
sequence is coded into a sequence of transmitter symbols, the random 
process which generates the messages and the operation of the coder may 
be combined to obtain the new random process which is the transmitter 
strategy. 

In Sect. 4, eq. (38) et seq., it was pointed out that the self-information 
of a symbol can on the average only be reduced by statistical dependence 
on preceding symbols. The same is true of the mutual information pro- 
vided by a received symbol about transmitted symbols for a channel with 
no storage. If the transmitter wants to maximize the average amount of 
mutual information received, he can do no better than to select successive 
transmitted symbols independently from some distribution p(ui). Then 
the problem of choosing a transmitter strategy reduces to the problem of 
choosing a first order distribution p(ui) for the transmitted symbols. Then 
p(ui) and the q(vj\ui) together determine channel operation completely. 
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Capacity of a Noisy Channel. The channel capacity C of a given noisy 
channel is defined as the maximum value of the transmission rate R which 
can be obtained by varying the transmitter strategy. For a channel with 
no storage, this is the maximum R which can be obtained by varying p{ui), 
with q(vj\ui) fixed. Thus from eq. (69), 

(70) C = max R = max ]T] X) P( u dq( v j\ u i) log — 3 — — [ » 

p(.u t ) p(m,) U=iy=i q(vj) J 

where the values of q(vj\ui) are held fixed as the p(ui) are varied, and the 
variation of q(vj) is determined from the relation 

(71) q(vj) = X) P(Vi)q(vj\ui). 

The maximization is carried out by differentiating eq. (70) for R with 
respect to each of the p(ui), subject to the constraint that 

(72) S p(ui) = 1. 

t=i 

However, this maximization may lead to negative values for some of the 
piui). It is then necessary to eliminate one or more of the input symbols 
by setting its probability at zero, and to maximize again with a smaller 
input set, until a maximum R is obtained with all p(ui) non-negative (30). 

Interpretations of Capacity 

One interpretation of the capacity C of a noisy channel is provided by 
its definition. In Example 1, Fig. 8, the binary erasure channel has a rate 
of transmission R = p when 0's and l's are transmitted with equal proba- 
bilities, as shown by eq. (63). This is the maximum rate attainable for 
this channel, and is therefore its capacity, C = p. One bit of information 
per symbol is supplied to the channel, and on the average p bits of informa- 
tion about the transmitted symbol are received. But the transmission process 
is not reliable. Since information is being supplied to the channel at a 
rate greater than the channel capacity, not all of it can get through, and 
the channel determines in a random fashion which bits will be lost and 
which will be saved. 

Feedback Interpretation. In the binary erasure channel, suppose that 
the transmitter can look over the receiver's shoulder and can see which of 
the transmitted symbols have been erased in the channel. This can be 
accomplished by having a noiseless feedback channel from receiver to 
transmitter. Every time a transmitted digit is erased the transmitter can 
repeat it, going on to the next digit as soon as the first unerased version 
of the preceding one has been received. 
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The transmitter is now supplying the channel with information at an 
average rate of p bits per transmitted symbol. The repeated digits do 
not give additional information about the message to be transmitted, but 
only about where erasures have occurred in transmission. The receiver 
receives information at the same average rate of p bits per symbol, and 
receives each message symbol once. Here channel capacity has the inter- 
pretation it has in the case of a noiseless channel: it is the maximum 
average rate at which information can be transmitted reliably over the 
channel. In fact noiseless channel coding Theorem I: Controlled Source 
(Sect. 5) applies verbatim to this noisy channel. 

Coding Interpretation. A noiseless feedback channel is not usually 
available. Fortunately it turns out that it is not needed. It is possible to 
obtain reliable transmission over a noisy channel, at any rate less than 
channel capacity, by proper encoding, without making use of any feed- 
back information. This is the primary justification of "capacity" as a 
significant parameter for a noisy channel. Indeed it is perhaps the most 
important single justification for the definition of mutual information and 
the whole structure of information theory. The formal expression of this 
fact is the: 

Noisy Channel Coding Theorem. Given: a discrete source of type I, 
II, or III (Sect. 5) with average rate R t bits per second', a discrete noisy channel 
without storage of capacity C t bits per second; and any 8 > 0. If R t < C t , 
it is possible to encode sequences of source symbols for transmission over the 
channel so that the probability that such a sequence will be incorrectly decoded 
is < 5. This is not possible if R t > C t . 

Relation to Noiseless Case. This theorem is essentially the second 
Noiseless Channel Coding Theorem, with a few minor modifications. Both 
of these theorems may be strengthened to give relations between the error 
probability 8, the difference C t — Rt between rate and capacity, and the 
length L of the sequence of source symbols which must be encoded. Results 
like the Tchebysheff Inequality and the Central Limit Theorem must be 
applied to both the source self-information distribution and the channel 
mutual information distribution. Some work has been done on these 
problems recently (Refs. 31-34). 

Implications. The Noisy Channel Coding Theorem shows that a lack 
of reliability in a channel does not impose a corresponding lack of reliability 
on the received and decoded messages. This alone is not surprising. For 
example in the binary erasure channel, transmitting at rate R = 1/N bits 
per symbol by repeating each message binit N times for transmission, the 
error probability per message binit is 

(73) gN = {i ir) = 2-0-IR) log (I/a) 
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since the receiver can decode each message binit unless all N repetitions 
are erased. The probability in eq. (73) can be made arbitrarily small, 
but only by letting R — > 0. 

The theorem also states, however, that error probability can be made 
arbitrarily small without decreasing rate R, so long as R < C, the channel 
capacity. This requires proper encoding of long sequences of source sym- 
bols. 

Construction of Codes. There is as yet no analog to the Huffman 
code for noisy channels. Considerable work has been done in designing 
codes for the binary symmetric channel of Fig. 9 (Refs. 35-43). However, 
no simple, explicit coding procedure has yet been found for transmitting 
at rates arbitrarily close to channel capacity with arbitrarily small error 
probability. 

The only constructive procedure available transmits at rates less than 
capacity. However, if the rate is kept fixed, it is possible for the receiver 
to set the error probability as low as he desires, but this depends on how 
much delay he is willing to tolerate. This procedure has been discussed 
for the binary symmetric channel (Ref. 40). It will be illustrated here for 
the binary erasure channel of Fig. 8. 

Error-Free Coding for the Binary Erasure Channel 

In the binary erasure channel of Fig. 8 the error probability can be re- 
duced by using some of the input binits as information symbols and some 
as check symbols. Such a coding procedure is illustrated in Fig. 10. 
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Fig. 10. Parity check coding for the binary erasure channel. 
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Parity Check Codes. Assume that the probability q of erasure per 
digit in the channel is 0.05. In Fig. 10, each group of four successive 
message digits has added to it by the coder a fifth digit for checking. The 
added digit is selected to be a or a 1 so as to make the total number of 
l's in the block of five coded digits even. A check digit of this type is 
called a parity check (Ref. 35). If the channel erases only one digit or none 
in each block of five, the receiver can correct the erasure by filling it in 
with a or 1 so as to make the total number of l's in the block even again. 

The channel may erase two or more digits in a single block of five, as 
shown in the third block in Fig. 10. The receiver cannot correct that 
block. However, with q = 0.05, so that on the average only 5 per cent of 
the transmitted symbols are erased, the receiver will be able to correct 
more than three-quarters of the erasures. In fact, the average number of 
erasures per block of N = 5 remaining after correction is equal to Nq, 
the average number before correction, less Np N ~ l q, the contribution to 
the average due to blocks containing a single erasure. This gives 

(74) Nq - Np N ~ x q = Nq[l - p^ 1 ] = Nq[l - (1 - q)*' 1 ] 

< Nq[l - (1 - q) N ] < Nq[l - (1 - Nq)} = (Nq) 2 . 

Thus the number of erasures remaining is reduced from Nq to (Nq) 2 , by 
a factor of Nq = 5(0.05) = 0.25. 

Behavior of First Stage. In the coding procedure illustrated in Fig. 10, 
input information is being supplied at an average rate of % = 0.80 bit 
per input binit. The capacity of the channel is C = p = 1 — q = 0.95 
bit per symbol. The resultant probability of remaining erasures is <^<t 
of the original probability of erasure in the channel. Without reducing 
the input information rate it is possible to reduce the probability of remain- 
ing erasures by iterating this kind of checking procedure. The second 
step in such an iteration is illustrated by Fig. 11. 

Iteration. In Fig. 1 1 , the size of the basic block has been increased from 
5 to iVi = 10 digits, the first nine being message digits and the tenth 
being a parity check on the preceding nine. After nineteen such blocks 
of ten, the coder adds a twentieth check block. The first digit in the check 
block is a parity check on the first digits in each of the nineteen preceding 
blocks, i.e., digits 1, 11, 21, • • •, 181 in time order. The second digit in the 
check block checks all the second digits in preceding blocks and so on; 
the last digit in the check block checks the nine preceding check digits, 
and it is in fact a parity check on the whole group of (10) (20) = N1N2 
= 200 digits. This is visualized most easily as in Fig. 10, in which the 
blocks are aligned below one another, and each digit in the check block 



INFORMATION THEORY 



16-37 



checks the column above it. The last digit in the check block checks 
both the row to its left and the column above it. 

The receiver decodes each row which has no more than a single erasure 
as soon as the check digit at the end of that row is received. If a row has 
more than one erasure, it can still be decoded properly when the check 
block arrives, if each column has only one erasure left after the first de- 
coding step. 
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Fig. 11. Iterated checking for the binary erasure channel. 



Erasure Probability. Since none of the digits appearing in a single 
column have ever been together in a check group before, they are sta- 
tistically independent, and the distribution of erasures in each column is 
binomial again. Define q as the erasure probability in the channel, q± as the 
average erasure probability remaining after correction of rows, q 2 as the 
average erasure probability remaining after checking by columns, pi = 
1 - ?i, V2 = 1 - q%- 



(75) 



N iqi = N lQ - N lV N - X q < (Niq) 2 , <Zi < {N,q)q, 



N 2 q 2 = N 2 qi - N 2 pi~ 1 qi < (N 2 q x y, q 2 < (N 2 qi)qi 



Forg = 0.05, Ni = 10, this gives qi < |g. ForiV 2 = 20, q 2 < \q x . 

Further Iteration. The next step, keeping Ni = 10 and N 2 = 20, is 
to add a check layer of 200 check digits after 39 layers of 20 blocks of ten 
digits each have been transmitted. This third order check will again 
multiply the erasure probability by a factor <3^. This procedure can 
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be repeated indefinitely, giving for the kth order check a remaining erasure 
probability 

Qk < (N k q k -i)qk-i < fefc-i, or ?* < 2_ V 

( 76 ) 

N k = 2N h - X = 2*-^i. 

In the limit as k — > . oo, the remaining erasure probability becomes arbi- 
trarily small. The rate of transmission in bits per symbol is just the 
fraction of input symbols which are message digits and not check digits. 
This is 

R = (i-A)(i -*)(!-*)••• 

> 1 — (To + 20 "T TO + ' ' ' ) 

>i-d 3 o)(i + i + i + ---) 

> 1 - & = 0.80. 

Thus the rate is at least as great as the rate in the simple block checking 
scheme of Fig. 10, but the error probability is as low as the receiver cares 
to set it if the transmitter adds the check digits of all orders, and if the 
receiver is willing to wait long enough for a sufficiently high order check to 
come along before decoding. 

Relation between Error Probability, Rate, and Delay. The iter- 
ative coding procedure just discussed is not optimum. However, it shares 
two characteristics with optimum systems. 

1. The reliability attained increases, for fixed rate, as the permitted 
coding delay increases. 

2. The reliability attained increases, for fixed delay, as the required 
transmission rate decreases. 

For any noisy channel there is a trading relationship between the proba- 
bility P e of residual error, the permissible delay N, the transmission rate 
R, and the channel capacity C. Here N is the number of symbols delay 
permitted between the transmission of a given symbol and the computation 
of its decoded version. The best terms of trade can be shown to give an 
approximately exponential decrease of error probability with delay : 

(78) P e = e - X( - C ' R)N , 
in the sense that 

/ log PA 

(79) lim(--|— ) = x(C,R) 

exists f or C > R as a positive number. The function x(C, R) is called the 
exponent of the error probability. For C — R small but positive, the ex- 
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ponent is approximately given by 



(80) 



x(C, R) = 



go - xy 

2<rj 2 



where 07 2 is the variance of the mutual information distribution for the 
given channel, and for the transmitter distribution p(ui) which attains 
capacity (Refs. 31, 32, 34, 41). 

8. THE CONTINUOUS CASE 

Continuous Sources 

A waveform like that of Fig. 12 shows two kinds of continuity. It takes 
on a continuum of amplitude values, and its amplitude changes con- 
tinuously with time. 

Signal 
amplitude 




A continuous waveform. 



Quantization. The amplitude continuity may be removed by ampli- 
tude quantization, as in Fig. 13. This may be accomplished by a quantizer, 
which has an amplitude transfer characteristic of the staircase type as 
illustrated. The output is a waveform whose amplitude values are selected 



Quantizing t 
levels 




Fig. 13. An amplitude-quantized waveform. 



from a discrete set, but whose jumps from one value to another occur at 
arbitrary times. The difference between the input signal and the quantized 
output is the quantization noise (Ref. 44). 

Sampling. The time continuity may be removed by making periodic 
observations of the amplitude of the waveform, deriving from the con- 
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tinuous time function a sequence of sample values. The period of the 
sampling is called the sampling interval. The resultant samples still have 
amplitudes selected from a continuous set. 

Sampling Theorem. If a waveform x(t) is bandlimited to frequencies 
between and W cycles per second, then it is completely determined by its 
samples x(kT) taken at a sampling interval T = 1/(2W) seconds. The func- 
tion x(t) may be re-created from its sample values by the expansion 



(81) 



* sin (t - kT) 

x(t) = 2 *W - 

*« (Jt-kT) 



Only bandlimited signals will be discussed here. The expansion eq. 
(81) makes it possible to replace all such functions by sequences of sample 
values (Ref. 45). Some work has been done recently on the nonband- 
limited case (Ref. 46) . 

Sampled and Quantized Signals. A waveform which has been 
sampled and quantized produces a sequence of sample values, each of 
which has been selected from a discrete set. Such a waveform is illustrated 
in Fig. 14. Definitions of self -information and source rate carry over to 
such a sample sequence directly from the discrete case. If the waveform 
being sampled is very smooth, the redundancy present in the sample 
sequence may have a particularly simple structure, and simple conditional 
coding procedures may be quite efficient in removing it (Ref. 47.) 
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-s- Time 



Fig. 14. Time samples from an amplitude-quantized waveform. 

Binary Pulse-Code Modulation. If a function X(t), bandlimited to 
W cycles per second, is quantized to M = 2 m levels, the sampled and 
quantized output consists of selections from a set of M = 2 m possible 
sample values. Each level may be represented uniquely by a sequence of 
m binary digits, giving the number of the level in binary notation. The 
binary number may then be encoded for transmission as a sequence of 
pulses and spaces, and transmitted in a bandwidth of mW cycles per second. 
This process is illustrated in Fig. 15. The resulting signal, although re- 
quiring more bandwidth than the original, may be transmitted over a noisier 
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channel, since the receiver need distinguish only the presence or absence of a 
pidse (Ref. 48). 
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Fig. 15. Binary pulse — code modulation. 

Self-Information of Continuous Signals. In a sampled sequence 
which is not quantized, each sample value is selected from an infinite set 
and may have infinite self -information. If the samples x(kT) are statis- 
tically independent and have the probability density p(x), then — log p{x) 
and its average value 



(82) 



H(X) = -f_ 



p{x) log p(x) dx 



are still definable in many cases, but they no longer represent information 
values. The quantity H(X) is still called the entropy of the distribution 
with density p(x), but is no longer the average self-information of the 
source per sample. The entropy function in the continuous case is not 
invariant under a change in the scale by which the amplitude x is measured. 
The infinite self-information associated with a selection from a con- 
tinuous set arises from the fact that the selection of a single real number 
between and 1 is equivalent to the selection of an infinite sequence of 
binary digits, namely the binary expansion of the real number, and con- 
versely. 

Continuous Noisy Channels 

Only stationary channels, without storage, and with bandwidth limited 
from to W cycles per second will be considered. Such a channel is 
defined by a conditional probability density function q(y\x). For a given 
value x of the transmitted sample, q(y\x) gives the density of the distribu- 
tion of possible received values y. 
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Example. Additive Noise. 

Let z be a noise voltage selected with probability density r(z), and let 
the received signal y be the sum of the transmitted signal and the noise, 
y = x -f- z. Then 

(83) q(y\x) = q(x + z\x) = r(z) = r(y - re). 

Thus a continuous channel with bandwidth W and additive noise is com- 
pletely specified by the distribution of the noise which is added. 

Mutual Information and Rate. If each transmitted sample value 
x is selected from a probability density p(x), and the channel is specified 
by the conditional density q(y\x), the joint density p(x, y) = p(x)q(y\x) 
defines both the channel and the transmitter strategy, in analogy to the 
discrete case. The probability density q(y) of the received sample values 
is then given by 

(84) q{y) = I p{x, y) dx. 
The random variable 

(85) I{x;y) = log^f- 

p{x)q{y) 

is again defined as the mutual information between x and y, and its average 
value 

r M r 00 PO, V) 

(86) R = I(X; Y) = p(x, y) log ^f- dx dy 

J-nJ-ao P(x)q(y) 

is the average rate of transmission of mutual information, in bits per 
sample. This measure retains its informational significance while self- 
information does not, because mutual information is invariant to a change 
in the scale on which both the transmitted sample x and the received 
sample y are measured. 

Example. Additive Noise. As before, let y = x + z, where z is an 
added noise, statistically independent of x. Then by eqs. (83) and (86), 

r 00 r x q(y\x) 

(87) R = I(X; Y) = p(x, y) log ^^ dx dy 

J-caJ-oo q(y) 

r" r™ r{x - y) 

= I I P(x, V) log — — dx dy 

J-nJ^ q(y) 

= - I I p(x, y) log q(y) dx dy 

I p(x, y) log r(y — x) dx dy 



= H(Y) - H(Z). 



— 00" — 00 
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Channel Capacity. The capacity C of a noisy continuous channel is the 
maximum value of the rate R which may be obtained in eq. (86) by varying 
the probability density p(x), with which the transmitted sample values 
are chosen. The variation is usually constrained so as to keep constant 
a given peak or mean square value of the time function x(t), which is 
determined through eq. (81) by the sample values. In general, finding C 
is a difficult variational problem. 

Example. Additive Noise. By eq. (87), the rate R in a channel with 
independent additive noise is the difference between the entropy of the 
distribution of received signal values y and the entropy of the distribution 
of the additive noise z. For a given channel, the noise distribution is fixed, 
so that maximizing the rate reduces to the maximization of H(Y) by 
variation of p(x): thus from eqs. (83) and (84), 



(88) max H(Y) = max \ — J j p(x)r(y — x) dx 



oo r /.oo 

X 



[/: 



log I p{x)r{y — x) dx 



dy 



subject to constraints on peak or average transmitter power. This problem 
is difficult. 

Entropy of Gaussian Distribution. If the sample values x{kT) of 
a bandlimited function x(t) are selected with statistical independence from 
a distribution with density p(x), the mean square value of the time function 
x(t) is equal to the mean square value of the samples (Ref. 44), which is 
given by 



(89) 



= I x 2 p{x) dx. 

J — on 



Thus x 2 is the signal power S. For x 2 = S fixed, the distribution with 
maximum entropy is Gaussian (Ref. 16), with 

(90) p{x) = (l/V2ri3)e- x2 ' 2S , 
and entropy, from eq. (82), given by 

/ — r x j — 

(91) H(X) = - (1/V2tS ) I e-* l/2S (log [1/ V2mS ] - x 2 /2S) dx 

•^—co 

= log V2^S + | log e 
= |log27re£. 
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Gaussian Additive Noise. If an added bandlimited noise z is statis- 
tically independent of the signal x, and either x or z has zero mean value, 



(92) 



y* = X 2 + z 2 = S + N, 



where N is the mean noise power. Thus if the channel is given and the 
average transmitter power is constrained, the received power is determined 
by eq. (92). The entropy H{Y) in eq. (87) will then be maximized if 
q(y) is Gaussian with variance S + N. 

However, the sum of two independent random variables cannot have a 
Gaussian distribution unless both random variables themselves have 
Gaussian distributions (Ref. 49). Thus only if the noise is Gaussian can 
the transmitter select a (Gaussian) p(x) which will lead to a maximum 
H(Y). The rate will then be 



(93) 



R = H(Y) - H{Z) 

= % log 2ttOS + N) -i log 2tdS 
— "I" l°g (1 + S/N) bits per sample. 



Rewriting eq. (93) on a bits-per-second basis gives: 

Capacity of a Channel with Additive Gaussian Noise. Given a 
channel bandlimited from to W cycles per second, with an average transmitter 
power S, perturbed by additive white Gaussian noise of total power N, its 
capacity is 

(94) C = W log (1 + S/N) bits per second. 

The restriction to white noise (noise which has a uniform spectral density 
in the interval to W cycles) is required in order that successive samples 
of noise be statistically independent. If they are not, the capacity will 
be greater than that given by eq. (94). 

Dependence of Capacity on Bandwidth. Holding S fixed and in- 
creasing W, the noise power iV increases with W, since noise power in 
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Fig. 16. Channel capacity and bandwidth. 
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frequencies previously rejected now enters the channel. For thermal 
noise and shot noise, the noise power N is directly proportional to W: 

(95) N = N W watts, 

where No is the noise power per cycle bandwidth. 
Substituting this in eq. (94) gives 

(96) C = W log (I +S/N W), 
which is plotted in Fig. 16. 

Interpretations of Capacity 

Some interpretation of the capacity of a continuous noisy channel is 
required, since the channel can accept input information at an infinite 
rate, but can only transmit information about its input to the receiver at a 
finite rate. 

Discrete Input Interpretation. One interpretation is provided by 
the fact that the noisy channel coding theorem still applies to the con- 
tinuous noisy channel. The output of a discrete source may be encoded 
for transmission over a continuous noisy channel at any rate less than 
channel capacity, and the receiver can then decode the received signal 
with arbitrarily small error probability. In this case the transmitted 
signals may be continuous waveforms, but they are selected from a finite 
set, and therefore have finite self-information (Ref. 45). 

Quantization Interpretation. If the receiver cannot distinguish 
between transmitted waveforms which are very near to one another, it 
is not necessary to transmit the precise waveform generated by a continuous 
source: some "near-by" waveform will do. The quantization process 
discussed earlier shows one procedure for selecting a near-by waveform. 
The transmitted waveforms of any finite duration are then a discrete set, 
and one may be selected by the receiver with small error probability 
despite the noisy channel. Other measures of distance, or fidelity of re- 
production, have been introduced and studied (Refs. 16, 46). 

Reduction of Ignorance Interpretation. A final interpretation also 
carries over from the discrete case. If successive samples are statistically 
independent, the receiver knows a priori that x will be selected from p(x). 
A posteriori the true value of x is selected from p(x\y),& narrower distribu- 
tion with less entropy. The change in entropy, 

(97) H(X) - H{X\ Y) 

measures the average reduction in the receiver's ignorance of the value of 
the transmitted sample (Refs. 16, 29). 
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Generalizations. The analysis of the continuous case has been ex- 
tended to cases of mixed type, i.e., to distributions which have discrete 
probabilities as well as densities (Refs. 16, 50, 51), and some discussion 
has been given of the nonbandlimited case (Ref. 46). 
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1. DEFINITIONS: SMOOTHING AND PREDICTION. SYMBOLS 

Time Sequence of Data. A plot is shown in Fig. 1 of a small portion of 
a time sequence of data, f(t) . Such a time sequence may also be represented 
by an electrical signal, in which the variable is a voltage or current. 

The sequence of data may be taken only at successive discrete intervals 
of time, instead of continuously. This is illustrated by the discrete ordinates 
indicated in Fig. 1. 



Amplitude 



Fig. 1. 
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Stationary Time Sequence of Data. The variation of a physical 
quantity with time constitutes a continuous time sequence of data. The 
values form a distribution. If this distribution does not show a long-range 
trend with time, the time sequence is said to be stationary. (See Chap. 13.) 

Quasi-stationary time sequence is a distribution that is statistically 
stationary (i.e., shows no trend) in the short range but not in the long 
range. 

Errors or Noise. There is usually a random error in the determination 
of a given physical quantity, or in its representation by a given electrical 




Amplitude 



Time 
Fig. 2. Variation of a physical quantity, with superposed error, with time. 



signal. This is illustrated by the erratic solid line of Fig. 2. The dotted 
line is the same plot as Fig. 1. 

A random error may be considered as added to the actual physical 
quantity. If there is no long time trend in the distribution of the random 
error, it is also a stationary sequence. 

In electrical signals the added sequence is usually called "noise," and 
in industrial processes, "disturbances." 

Smoothing Problem. For data with random errors such as Fig. 2, an 
averaging process could be used to reduce the error. This assumes that 
the physical sequence is "smoother" than the data sequence. In an elec- 
trical signal representing the data such an averaging process can be carried 
out by a filter. 

Optimum Filter. There is likely to be some filter design which has an 
optimum frequency-response characteristic. If the filter suppresses the 
rapid departures too much, it also suppresses some real variations in the 
physical quantity represented by the data. If, on the other hand, it does 
not suppress them sufficiently, it is not reducing the error as much as is 
feasible. For a classical theoretical analysis by Wiener see Ref. 1. The 
optimum filter as designed by mathematical theory is not usually critical. 
In practice an elementary filter is generally devised which approximates 
it and gives almost equal performance. 
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Predicting Problem. It is occasionally desirable not only to smooth 
the data, but also to extrapolate or predict the data. For example in Fig. 
2 it may be needed, at time t, to predict the most likely signal which will 
occur at time t -f- T. This is feasible because the variation of the physical 
quantity is restrained by physical laws, and it does not have complete or 
random liberty of action. 

Wiener's analysis indicates that prediction may also be effected with a 
filter. An optimum design is secured from nearly the same formulation 
as that used for the smoothing process. 

0.6 
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Fig. 3. Amplitude and phase characteristics vs. frequency of an optimum filter. 

Example. Prediction Filter. The transfer amplitude response and 
transfer phase shift of a smoothing and predicting filter designed according 
to the Wiener theory are presented in Fig. 3. The ordinates are shown as 
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Fig. 4. Signal and noise power spectra. 
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functions of the radian frequency co. The filter is designed for a prediction 
time of 1 second, and for the signal and noise power spectrum illustrated 
in Fig. 4. More details regarding the problem and the design are given 
below. 

Prediction: Discrete Data. When the data are discrete rather than 
continuous, the solution is not simply embodied in the form of an electrical 
filter. The solution describes instead a mathematical process that ac- 
complishes an analogous averaging and predicting effect. 

Basis of Treatment. In the present exposition, the Wiener theory is 
followed (Ref. 1). Advantage is taken of the treatments of Levinson 
(Ref. 2) and of Bode and Shannon (Ref. 3) in simplifying the presentation. 

It is to be recognized that much other work has been done on the subject. 
At the close of the chapter some of this work, particularly recent develop- 
ment, is noted. 

Symbols 



A, B constants 

A n numerator of partial fraction 

A(ui) amplitude part of transfer response, in nepers 

a, b, c portions of an integral 

B(oi) phase of transfer response, in radians 

bj coefficient of p 1 in polynomial 

C parameter 

C, D, E, F numerators of partial fractions, sometimes with subscript 
indices 

Cj capacitance of jth element in filter 

e base of Napierian logarithms 

F(w) signal correlation function 

/, f(t) signal amplitude 

g, g(t) noise amplitude 

gj coefficient of p in continued fraction 

h(t) total instantaneous wave amplitude 

I, II integrals, forming part of more extensive formulas 

i V^l 

I(w) current, function of frequency 

/, H limits of summation indices 

j, h summation indices 

K(oi) transfer response function of frequency, of Wiener filter 

Ko(x) Bessel function of second kind, pure imaginary argument 

k(t) transfer response function of time, of Wiener filter 

Lj inductance of jth element in filter 

M , N limits of summation indices 

m, n indices 

p Heaviside operator = iw 

Q((o) factor of *(w), with zero phase angle 
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q(t) Fourier transform of Q(cS) 

R resistance 

s variable of integration, for radian frequency 

T prediction time 

t time variable 

u variable of integration, for radian frequency 

V, V(ui) voltage, function of frequency 

v, v(t) voltage, function of time 

vo voltage amplitude 

X, Y numerators of partial fractions 

F(w) transfer response function of frequency 

F(w) admittance, function of frequency 

Z(w) impedance, function of frequency 

Z\{p) driving point impedance, function of operator p 

Zx(p) transfer impedance, function of operator p 

ZtW) transfer impedance, function of frequency 

a\, «2 • • • /3i, 182 • • • zeros of polynomials, with imaginary parts > 

«i*> «2* • • • /3i*, 02* • • • complex conjugates of corresponding a's and /3's 

e constant 

integration limit on time variable 

k constant 

X(w) natural logarithm of *(co) 

n, v indices 

t time variable, for correlation or integration 

3>(co) correlation spectrum, Fourier transform of 4>(r) 

<I> n (w) correlation spectrum of nth derivative of input function 

<£//•(«), $gg(co) autocorrelation spectra 

<j> phase shift 

<t>(r) correlation function 

</>„(t) correlation function of nth derivative of input function 

0//(t)> ^ggO") autocorrelation functions 

0/g(r), <p g f(r) cross-correlation functions 

\I>(co) factor of <£(w) for which singularities have imaginary parts 
>0 

^'*(w) complex conjugate of ^(co) 

^ n (w), ^ n *(w) corresponding factors of ^ n (cS) 

^(t)) ^*W Fourier transforms of ^(co), ^>*(w) 

w radian frequency 

wj zero of polynomial, with imaginary part > 

1 coy* complex conjugate of coy 

2. DEFINITIONS: CORRELATION 

Autocorrelation Function. The amplitude of a signal at any given 
time is not wholly independent of its value at other times. The cor- 
relation that exists may be expressed in terms of an autocorrelation function. 
In Fig. 5 the signal amplitude / is measured at times t and t -f r. The 
autocorrelation function 4>{t) of a signal is the average product of the 
signal at time t and the signal at time t + r, averaged over a period of 
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f(t + T). 




Time — > 
Fig. 5. Data taken for determination of autocorrelation coefficient. 

time long enough to smooth out instantaneous fluctuations. With the 
overscribed bar to indicate averaging, 



*(r) = /(«)/(« + t), 



or 



(1) 



</>(r) = lim 



i r e 



+ t) dt, 



where t = finite time shift. 

The autocorrelation function is a measure of the extent to which the 
value of f(t) at any given time can be used to predict f(t) at a time interval 
r later. 

Example. An autocorrelation function is illustrated in Fig. 6. This is 
the one assumed for the signal whose spectrum is illustrated in Fig. 4. 




•2 -1 

Fig. 6. Autocorrelation function. 



1 

Correlation time, t 
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The autocorrelation function in general is at a maximum at r = 0, 
and it usually drops off to zero or nearly zero for large values of r. It 
shows even symmetry about r = 0. The peak is broad when f(t) contains 
primarily low frequencies and narrow when f{t) contains primarily high 
frequencies. 

Power Density Spectrum. It is useful to deal with the Fourier trans- 
form of the autocorrelation function 0(t) which may be called the auto- 
correlation spectrum, <J>(co). This is 

(2) $(«) = f 0(r)e- iur dr, 

and reciprocally 

1 r°° 

(3) 4>{t) = — I $(a)e mT dco. 

2tt J_ o0 

Wiener (Refs. 1 and 4) has identified the autocorrelation spectrum 4>(co) 
with the power density spectrum of the signal (with a suitable normalizing 




Radian frequency, u 
Fig. 7. Autocorrelation spectrum. 

factor). That is, the function $(w) is a measure of the mean power in the 
signal per unit of band width for each frequency. 

The power density spectrum for the autocorrelation function of Fig. 6 
is shown in Fig. 7. For simplicity a normalizing factor, to be discussed 
later, is introduced. This has the effect of causing both plots to reach a 
maximum ordinate of 1. Both plots show functions having even symmetry. 

The power density is expressed in terms of coefficients of sine and cosine 
terms of positive frequencies. The coefficient, for any given frequency, 
is twice the numerical value of $(oo) of eq. (3) for that frequency. This 
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corresponds to the addition of <£(— co) to <3?(w). This factor represents part 
of the arbitrary factor mentioned in the paragraph above. 

Autocorrelation of Noise. In the same way as the signal, the noise 
shows autocorrelation as denned by eq. (1) and illustrated in Figs. 5 and 6. 

For the purposes of the illustration in Fig. 4, the autocorrelation of 
the noise is assumed to extend over a shorter time than for the signal, 
i.e., over only a negligible interval of time about r = 0. 

The autocorrelation function for the noise also has a Fourier transform 
as defined by eq. (2) and illustrated in Fig. 7. In the example of Fig. 4 
the assumption just mentioned greatly stretches the frequency scale of 
the noise curve in Fig. 7. Thus the power spectral density has been taken 
as substantially a constant over the frequency range of interest. 

The cross-correlation function between two different signals is similarly 
defined as the average product of one signal g(t) and a second signal fit) 
at time t -\- r. Thus 



0/*(t) = fit + r)g(t), 

1 r e 
<t> fg (r) = lim - f(t + r)g(t) 
o— * oo 2d J _« 



or 

(4) 4> fe (r) = lim - | f(t 4- T )ait) dt 

The cross-correlation in the reverse sense is 



(5) 



4>gfir) = <t>fg{-r) 



Examples. If as in Fig. 8 the signal/ is measured at time t + r and the 
noise g at time t, eq. (5) is the cross-correlation between signal and noise. 
If fit) represents an input to a system and git) its output, then <p/ g rep- 
resents the cross-correlation of input and response. 




Fig. 8. Data taken for determination of cross-correlation coefficient. 
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Resultant Correlations in Signal Mixed with Noise. The auto- 
correlation $(t) of the function h(t) = f(t) + g(t) is made up of the auto- 
correlations of the component functions that may be denoted respectively 
by 0//(r) and <£ gg (r), and the cross-correlations. Thus 

(6) <t>(r) = 0//(r) + <t> fg (r) + g/ (-r) + gg (r). 

In most practical systems there is no correlation between signal f and noise g, 
and also the average value of the noise is zero. Thus 

*/*(r) = 0, g /(-r) = 0, 

and 

(7) 0(r) = 0„(t) + *„(t). 

Cross-Correlation Spectrum. The corresponding relations hold for 
the Fourier transforms, and the cross-correlation spectrum is 

(8) $(«) = $//(«) + $ /g (co) + $ g/ (co) + *„(«). 
In most practical systems, as discussed above 

(9) *(«) = *„(co) + $ gg (co). 

This last equation means that the power density spectra of the signal 
and of the noise add to form that of the total wave. This addition has 
been performed in Fig. 4. 

3. RELATIONSHIP BETWEEN CORRELATION AND SIGNAL STRUCTURE 

Latitude in Interpretation of Power Spectrum. In general, there 
is a unique relation between a function and its Fourier transform. The 
power density spectrum of the signal does not, however, define a unique 
signal because it gives no phase information for the spectral components. 
Thus the power density spectrum imposes restrictions only on the signal, 
and it does not define it. 

Construction of Signal from Bandlimited or Shaped Impulses. 
The signal may be considered as originating from a succession of idealized 
impulses sent out according to the information contained in the message. 
This succession of impulses is then transmitted through a bandlimiting or 
shaping network. The output of the network becomes the signal. 

The character of the shaping network affects the power density of the 
spectrum. Consider an especially simple specific case (Ref. 5). Informa- 
tion is assumed as coming at periodic intervals. It is further assumed that 
the signal is of sufficiently random character to have an average amplitude 
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of zero and to give a constant power density over the spectrum. In this 
simple case the power density spectrum is simply the square of the transfer 
amplitude response spectrum of the shaping network. The corresponding 
amplitude response curve of Fig. 7 is shown in Fig. 9. 



1.5 

1.0 

0.5 
















' 













12 3 4 

Radian frequency, co 

Fig. 9. Amplitude vs. frequency response of shaping network. 

Influence of Phase Response of Shaping Network. The square of 
the amplitude response does not as yet specify the phase characteristic of 
the shaping network. One may simply assume this phase shift to be zero 
throughout the spectrum. In such a case the Fourier transform of the 
amplitude characteristic of Fig. 9 would be that illustrated in Fig. 10. 
This gives the output response of such a network to a unit idealized impulse 
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Fig. 10. Transient response of hypothetical shaping network having zero phase shift. 
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signal input at time t = 0. It shows a response before the occurrence of 
the impulse, which indicates that the particular network is not physically 
realizable. 




12 3 4 

Radian frequency, to 

Fig. 11. Phase shift of physically realizable shaping network. 



For the network to be physically realizable it must at least meet the minimum 
phase condition (Ref. 6). For a transfer response spectrum of 

(10) F(co) = e AM+iBM 



the minimum phase condition is that 
(11) B( a ) = - 

7T ^0 



2co f°°A(s) -A(to) 
s 2 -co 2 



ds. 



The phase shift of any realizable network is equal to or greater than this. 
The phase shift meeting the condition for the amplitude response charac- 
teristic of Fig. 9 is plotted in Fig. 11. 

For a response having the amplitude characteristic of Fig. 9 and the 
phase of Fig. 11, the Fourier transform is presented in Fig. 12. This no 
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Fig. 12. Transient response of physically realizable shaping network. 
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longer shows an advance response to the impulse signal input. The response 
now starts at t = 0, the same time as the signal input. 

Representation of Network Properties in Complex Frequency 
Plane. If the square of the transfer amplitude response is a rational func- 
tion of co, it may be expressed thus: 



(12) |F(co)| 2 = <l>(co) =k 



2 (co — ai)(co — ai*)(cj — a 2 ) (co — (X2*) 
(co - ft) (a, - ft*)(co - ft)(co - ft*) 



where k = a constant, 

a, jS = zeros of numerator or denominator with imaginary parts > 0, 
a*, jS* = complex conjugates of a, ft 

The a's are the zeros, and the fts poles, of the function [ F(co) | 2 . A 
plot of these, for the function plotted in Fig. 7, is given in Fig. 13. There 
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Real part, co 
Fig. 13. Singularities of rational correlation function. 



are only poles, one being above, and the other below, the axis of real co's. 

The response indicated by Fig. 9, with zero phase shift, is not represented 
by a rational function, and hence it cannot be described simply by zeros 
and poles in the complex frequency plane. 

Location of Zeros and Poles for Minimum Phase Network. From 

Bode and Shannon (Ref. 3), the minimum phase network has the transfer 

response function 

(co — ai)(co — ao) • • • 

(13) F(co) = k- — 

(co - &)(« - &) ••• 

The zeros and poles are all in the upper half-plane of the complex fre- 
quency space. A plot of the zeros and poles which applies to the amplitude 
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response of Fig. 9 and phase shift of Fig. 11 is shown in Fig. 14. In this 
specific case there are no zeros and only one pole, which is in the upper 
half-plane. 
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Fig. 14. Singularities of physically realizable shaping network. 



4. DESIGN OF OPTIMUM FILTER 

Criterion of Optimization. A criterion of performance is necessary 
to judge when one filter design is better than another. That used by Wiener 
is based on a comparison between the filter output and the actual signal, 
freed of noise, at the extrapolated time. The difference, or error, is meas- 
ured as a function of time, and its root-mean-square value determined. 
The optimum filter is taken as that for which the root-mean-square error 
(Chap. 13) is a minimum. This assumption is important to the develop- 
ment of the theory. 

Wiener Solution, Smoothing and Prediction. The optimum filter 
proposed by Wiener has a frequency response which may be designated as 
K(u). This has a Fourier transform k(t). The reciprocal relations between 
the two are: 



(14) 



(15) 



K(a) = f k(t)i 

<J — on 



, — iu t 



dt 



k(t) 



i r™ 



do). 



In the simple Wiener solution the transfer response of the optimum filter 

has the value 

1 r 00 r™ $ff(u) 

(16) K(p>) = I e-^dt I -^^ e *«(«+r> du 
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Here $//(&>) represents, as in eq. (8), the autocorrelation spectrum of co 
of the signal alone. The co is changed to u as a variable of integration. 

The quantities ^(co) and ^*(co) represent the factors of the autocorrelation 
spectrum of the signal plus noise, $(co) of eqs. (8) and (9). The factors are 
taken, as in eqs. (12) and (13), such that 

(17) $(«) = *(co)**(co), 

where ^r(co) has zeros and poles only in the upper half-plane of the complex 
frequence space, and ^*(cd) only in the lower half-plane. The quantity T 
is the prediction time. 

In the limiting case where no noise is added to the signal $//(m) = <£(m) 
= ^{u)-^*{u). Thus in the integrand of eq. (16), the ratio ^f/(u)/^*(u) 
= V(u). 

In this limiting case eq. (16) takes on the simpler form 



1 r" ■ , r" 

2tt^(co) J J_ K 



,iu(t+T) 



(18) K(a) = e- l0,t dt\ V(u)e m Wdu. 

2tt^(co) J J_ K 

Factorization Problem. The key to the solution of the filter is of 
course the solution of the factorization problem expressed in eq. (17). 

The formal solution, in terms of the $(co) assumed known, is given by 
Levinson (Ref. 2) as 

* ( w ) = e X(w) 

i r x co log f»(s) 

(19) X(«) =- 2 \ ds. 

TT Jo CO — S 

(20) ^*(co) = $(w)/¥(co). 

Identification with Minimum Phase Network. The connection of 
eq. (17) with eqs. (12) and (13) identifies >J*(co) with the transfer response 
characteristic Y(co) of the minimum phase shaping network which has the 
amplitude response characteristic 



I F(co) | = Vifa). 

The eq. (19) shows some similarity to eq. (11). The two differ because 
eq. (11) determines the phase only of ^(co) (or Y(u) of eq. (10)) whereas 
eq. (19) determines the entire complex exponent in the equation just above 
it. 

Alternative Formulation. In actual practice eqs. (16) and (19) lead 
to substantial mathematical difficulties even in simple cases. One practical 
approach is, as in eq. (12), to take <£(co) as a rational function. When this 
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is done ^'(co) is given by eq. (13) and ^(co) by its complex conjugate. 
That is, 

M 

II (co — a m ) 

(21) ¥(«)=IC^ ; 

n (« - is B ) 

n=l 

jVf 
IJ (w - «m*) 

(22) **(co)= K ^-— • 

II(«-/8 B *) 

n=l 

A further transformation is convenient because integrands in the form 
of sums are more easy to handle than as products. 

The products may be expanded to partial fractions, thus, for first order 
poles 

N n 

*(») = £ -^V- 

n=l U> — (S n 

For poles of order n, 

N p 

*(«) = 2^ 

~1 (« - /S„)" 

In a similar manner the principal factor of the integrand at the right in 
eq. (16), if it has no poles of higher order than the first, may be expanded 
thus 



*,/(«) " Dj ' 'E ; 



V*(co) y = i co — coy y = i co — coy* 
Where poles exist of order ju, this becomes 



(24) 



"M\(u-Mi)" (co-coy*)"/ 



^*(co) y = i \(co - 



Wiener has shown that when these are substituted in eq. (16) the solution 
becomes, for the case with poles only of the first order in both $//(co)/^*(co) 
and ¥(w), 

(25) K( a ) = E -!— / S 



y=i w ~ /V n=l & — fi n 
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For poles of order /x, this is 



J ^ _1 (iTY 



7=1 k=0 v\(oi — co,-) M " 
(26) K( a ) = 3 — N — 



C 



IZi (« - jS„)" 

Empirical solutions of the factorization problem are mentioned in Sect. 5. 

Example 1. A simple illustration of prediction given by Wiener (Ref. 1) 
has been presented in Figs. 3 to 12. The equations for this illustration may 
now be noted. 

For the signal alone : 

(27) *,,(„),= ——. 

« + 1 

The Fourier transform is found in Campbell and Foster (Ref. 7). This 
is tabulated with the argument p = iu instead of to, so that 

(28) ■*/,(?) = -f—- 

V ~ 1 

From pair 444 (Ref. 7) 

(29) 0//(r) = K M 

This is represented in Fig. 6, ignoring the factor 3^. 
For the noise alone: 

e 2 

(30) $ gg (co) = > e 2 , 

' (co/C) 2 + 1 

where C — > «.. 

(31) $(co) = *„(«) + * gg (co) 

1 ! 



co 2 + 1 (co/C) 2 + 1 co 2 + 1 

(32) 0( T ) = i(e- |T| + 6Ce- |T/C| ). 

For the conditions illustrated in Figs. 9 and 10, where it is assumed 
that the phase response of the shaping network is zero (Sect. 3), 



(33) Q gg (co) = V *„(<*) = 

(34) Q gg (p) = 



VcTTT 
l 



Vi - v 
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From pair 558 (Ref.7), 

(35) q gg (r) = (l/7r)/C (|r|). 

Here K is the Bessel function of the second kind, and imaginary argument 
(the actual argument being i\r\). It is tabulated in Watson (Ref. 8). 
This is illustrated in Fig. 10. 

For the shaping network assumed as physically realizable: • 

(36) ¥>,(«) - 

(37) * ff (p) = 

p+1 

From pair 438 (Ref. 7), 

(38) ¥(<) = e~\ t>0. 
This is illustrated in Fig. 12: 



03 — I 
1 



(39) 


co -f- l 


(40) 


-1 

yJr ■■*('n\ — 


p-1 


From pair 


439 (Ref. 7), 


(41) 


*//*(<) = e l , t < 0. 


(42) 


x/ N ' e[co - (iVl + e 2 /e)] 

^r(co) = : , 




CO — I 




t J { e[co 4- fcVl + e 2 / e )] 

^*(co) = : . 

co + i 




%(co) 1 




**(co) €[co -:t'][co + (iVl + e 2 /e)] 




D E 

= - + —, — 



- i co + (iVl + e 2 /e) 
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where 

i ' 
E= e+ VT+7' 

De~ T /(u - i) 

-K(co) = , » 

e[co - (iVl + e 2 /e)]/(co - i) 

■ e- T 

(e 4- Vl+ e 2 )(Vl + e 2 4- da) 



(43) K( a ) = 



,—T 



exp [-t'tan -1 (eco/vT+1 2 )] 



+ Vl + e 2 Vl + e 2 + e 2 co : 



2. .2 



The form of eq. (43) indicates the nature of the variables in i£(co). The 
first factor depends only upon the prediction time T, and the second only 
upon the noise spectral density e 2 . The denominator of the third factor 
is a combined function of noise density and frequency. All three of these 
quantities are real and hence affect only the amplitude response of K(u>). 
The numerator of the third factor is complex. It has a modulus of unity 
and expresses the phase shift of K(co). The amplitude response and the 
phase are the two quantities plotted in Fig. 3. 

Example 2. A second illustration, by Wiener, assumes no noise. The 
function required of the filter is prediction. Consider 

1 
(44) $(«) = 



(co 2 + iy 



1 






(P 2 ~ I) 2 




From pair 433 (Ref. 7), 






*(0 = [(l*f+l)/4]e- w . 






(46) 






-1 -1 


*(p) = 


1 


'a i co ) — ■ ■ ■ » \f i co j — _ » 
(to - if (co 4- if 


(P + I) 2 


From pair 442 (Ref. 7), 






^(t) = te~\ t > 0. 
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From eq. (18) 
(47) 



1 r™ 
K(o) = — I W 
¥(w) Jo 

1 r w 

= 77T ( 



+ T)e- 



dt 



e~ T te m 



■ l + Te- T e- l )e 



dt 



(&> - *r r _e 



-1 



tTe 



-Tn 



(w - S 



(48) 



Jf (co) = e~ r (l + T) + iaTe 



—T 



This solution is also reached from the alternative formulation of eq. (26). 

The solution consists of a term independent of co and a term which 
comprises a differentiation of the input wave. This was also the case in 
the previous example, except that there the total band was limited by the 
denominator. In the present case the total band is infinite. 

In polar form the solution is 



(49) K(a) = e- T Vl 4- 2T + T 2 + o 2 T 2 exp {ztan -1 [coT/(l + T)]}. 

5. EXTENSIONS OF PROCEDURE 

Filters with Lag. On occasion the urgency in time of reproduction of a 
signal mixed with noise is not so great as the need for greatest feasible 
reduction of the noise in the reproduction. 

In such cases the prediction time T is advantageously changed into a 
lag, that is, T becomes negative. The optimum filter formula of eqs. (16) 
and (26) still holds generally, but there are difficulties in carrying out the 
second integration, with the lower limit of zero. 

Wiener (Ref. 1) suggests an approximation thus: 



(50) 



Ja>T 



1 + (ia>T/2v) 
1 - (iaT/2v). 



The single case is considered where the ratio of eq. (24) contains only 
one pole in the upper half-plane of co. This is assumed of the first order, at 
coi. A possible pole in the lower half-plane may be at co 2 *. Then 



(51) 



(52) 



%(to)e 
**(co) 



iuT 



F 



1 + {ionT/2v) 

.1 - (io> x T/2v)\ (co - coi)(w 
■A n X 



N 



+ 



"2*) 



+ 



„tTl [1 - (t«iT/2lO]" (tO - CO!) (CO - C0 2 *) 
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In eq. (52) the first summation term in the partial fraction expansion is 
approximated only to n = N < v. Then 

(53) K(o>) « ( E , A " ]n + -^-) A(«). 

\n=\ [1 - {lO)\T/2v)\ n CO — Oil/ / 

In the example which was illustrated by eqs. (36) to (43), Wiener gives 
the result as 

(54) 

T 2 /2 1 + ia 



[1 + (T/2)][(T/2)Vl + e 2 - 6] (Vl + e 2 + ««)[1 - (io>T/2)] 

- (T/2) 



+ 



[1 + (T/2)](e + Vl + e 2 )(Vl + e 2 + « w ) 



Where the noise level is high and the signal weak a simple formula is 
obtained for the optimum filter with lag. Let 

(55) */,(«) = eF(«), $ gg (co) = 1, 

(56) $(w) = 1 + eF(u), 

$(0)) H-eF(co)' 

(58) « «F(eo). 

T/ia^ ts, iAe optimum filtering in such a case merely follows the spectral 
power density weighting of the signal. This corresponds to conventional 
procedure. 

Input Functions with Correlation in a Derivative. On occasion the 
input function that is dealt with shows a strong correlation in one of its 
derivatives. By this it is meant that the correlation function is sustained 
significantly, out to a time of the order of the prediction time or longer. 

Examples. This condition occurs when the function represents one of 
the coordinates of the motion of a massive body. If the forces on this are 
not too large, the velocity can show correlation out to a substantial time. 
Another case is that of a massive body acted on by forces which change 
only by small amounts from instant to instant. In such a situation the 
acceleration can show correlation over a useful time range. 

Cases of this type may assume significant practical importance, and in 
such situations it may be useful to base the filter design on the correlation 
function in the appropriate derivative. The autocorrelation function of 
the nth derivative may be denoted by n (7"), and its Fourier transform by 
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<p n (co). In the same way that $(co) was factored above in eq. (17), $ n (co) 
may be factored as 

(59) $„(«) = *n(co)* n *("). 

Then the Wiener formula for the optimum filter is, for the prediction time T, 



(60) K{fS) = l + icoT + ..- + 



(n - 1) ! 

■* n {u)e iut 



+ €-"" <fc -^ X 

2tt^ w (co) J J-oo w M 

(e iuT - 1) - (ittT) 



[■ 



(w - 1)!. 



dw. 



This is the form of the equation in which no noise is assumed in the in- 
put. Where there is noise the \&n(u) in the integrand would be replaced 
by the expression <p/y(to)/SI>*(co) in eq. (16). 

Example. Assume that the correlation spectrum of eq. (44) describes 
the correlation in the second derivative of the input function and that no 
noise is present. That is, 

(61) * a (r) = [(|r|+l)/4]e- w , 

or 

Then the solution for the filter for the prediction time T, as given by eq. 
(60), is 

K( a ) = 1 + iaT + I e~ lut dt — — e lut (e lut - 1 - iuT) du. 

2tt^ 2 (w) Jo J -oo ur 

As in eq. (46), 

y 2 (o)) = 



Let 

— 1 r™ e lut 

2tt J.^ u 2 {u - i) 2 

iu(t+T) Aut -rp iut 



— 1 r T e* M ^" t " i; 
2tt J_«, Lm 2 (w - -0 2 



ciw 



! (w — *') 2 w2 (w - *) 2 w(w — i) 2 _ 
The integration of the third term of the integrand is accomplished by the 
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combination of pairs 210 and 442 (Ref. 7) and gives 

c = -Tie' 1 - Te~ l + T, t > 0. 

The integration of the first and second terms is accomplished by the 
further application of pair 210, and gives 

a = -(t+ T)e~ {t+T) - 2e- ( ' +r) - (t 4- T) 4- 2, t 4- T > 0, 

h = te- 1 4- 2e~ l 4- t - 2, t > 0. 
Thus 

I = a + 6 + c 

= -^-'(e- 7, 4- T - 1) - e-'^e- 77 4- 2e~ T + T - 2), t > 0. 
Continuing to the second integration of eq. (60) 



•'o 



le- lut dt 



= f (Ate"' 4- £e~V 
Jo 



-A iB 



(co — t) 2 co — i 
co 2 II 

^2(w) 

if(co) = 1 4- ^T 7 4- co 2 A 4- co 2 £ 4- io> 3 B. 
(62) if(co) = 1 4- iwT 4- (ico) 2 (7 7 e- :r 4- Se~ T 4- 2T 7 - 3) 

4- {iu?(Te- T 4- 2e- 7 ' 4- T 7 - 2). 

In this solution, there are three successive differentiations of the input 
wave to add to the constant term. As in the previous example, character- 
ized by eq. (48), the absence of noise in the assumptions leads to an infinite 
band in the filter. 

"Filters" for Discrete Data. The signal which has been assumed in 
the discussion up to this point has been continuous. However, it could be 
expressed as the amplitude of a succession of discrete pulses, such as the 
maximum daily temperatures at a given location. 

Where the data are discrete, in electrical form or not, they cannot be 
passed through a physical electrical filter to carry out the operations which 
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have been discussed. The operations can, however, be expressed in terms of 
mathematical processes, with only small changes from the previous description. 
For discrete data eq. (1) becomes 

I M 

(63) 0(n) = lim — — Z f(m)f(m + n), 

and eq. (2) becomes 

00 

(64) *(«) = Z <f>(n)e~ iun . 

n=— oo 

The factorization problem of eq. (17) is modified somewhat, because 
$(co) is more likely to be an empirically determined rather than an ana- 
lytically expressed quantity. 

Wiener (Ref. 1) has indicated various means for handling this problem; 
one method is the following. 

Equation (17) may be written: 

(65) log *(w) = log tf (w) + log **■(*>). 
Also log <£(co) may be expanded into a Fourier series, as 

—1 00 

(66) log$(co)= E a n e- iun + a + E «ne- fwB 

n= — oo n=l 

With <£, and therefore log $, real, the series shows symmetry between 
positive and negative n's. From the discussions regarding eqs. (10), 
(11), (12), and (17) to (20), and recognition that the first term at the 
right of eq. (66) shows no response at positive times (n > 0), and the third 
term no response at negative times (n < 0), the terms of eq. (66) may be 
identified with those of eq. (65). That is, 

00 

(ao/2) + £ a n e~ iun 
i 

are identified with log ^(w), and 

—l 
(ao/2) + E ane- iun 

— 00 

with log^*(co). 

This permits the computation of ^(co) and ^*(co) from the Fourier series 
of eq. (66). The Fourier series itself may be obtained empirically by 
numerical computation or by the use of a harmonic analyzer. 

Other empirical solutions of the factorization problem have been pre- 
sented. Some of these relate to the determination of the minimum phase 
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of a network which shows an empirical transfer amplitude response charac- 
teristic (Refs. 6 and 9). 

Additional empirical solutions for the factorization have been presented, 
related to other problems (Ref. 10). 

Alternatives to Wiener Criterion of Optimization. Lee (Ref. 11) 
has explored the possibilities of an alternative to the Wiener criterion. 
He minimizes the integral square cross-correlation error (integrated with 
respect to time). He expresses the conditions in terms of an autocorrela- 
tion of the autocorrelation function (as if the latter were itself a signal) 
and a cross-correlation of the autocorrelation function and the cross- 
correlation function. In these terms the specifications for the optimum 
filters are completely analogous to those of Wiener. 

Zadeh and also Middleton and van Meter have outlined possibilities in 
the design of an optimum filter which uses the methods of decision theory. 
(See Ref. 12.) This utilizes all the information known to the designer about 
the signal and the noise, and optimizes the decision. This optimum mini- 
mizes the integrated risk of wrong decisions. The results obtained are 
chiefly of conceptual value, because the computational problems are 
formidable even in relatively simple situations. 

Other authors have also applied criteria of or akin to those used in 
decision theory (Ref. 13). The procedures are particularly effective con- 
ceptually where the signal interpretation occurs under extremely unfavor- 
able noise conditions, such as for the signals from fringe areas of search 
radars. The paper of Middleton and van Meter (Ref. 12) contains a 
complete bibliography. 

Zadeh and Ragazzini (Ref. 14) have extended the Wiener theory to the 
case where the data are described by a nonstationary time series. Par- 
ticularly they assume its approximation by a polynomial of a given order 
in time, with unknown coefficients. Such cases have a certain practical 
interest. The treatment uses an approach suggested in a report by Bode, 
Blackman, and Shannon, in 1948, to the Research and Development 
Board. 

Chang (Ref. 15) has enlarged the Wiener criterion by considering also 
integrated squares of errors in the frequency domain. The integration 
further weights these errors according to arbitrary functions of the fre- 
quency. He develops two theorems, a minimization theorem and a separa- 
tion theorem. The first is an extension of the Wiener theorem in terms of 
frequency, and the second represents an alternative procedure to the 
factorization methods of Wiener. 

Still other authors (Ref. 16) have given consideration to extending the 
hypotheses under which the Wiener criterion can be used. In particular 
they have extended the treatment to include nonstationary noise and a 
system having time-varying linear parameters. 
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Nonlinear Prediction. The discussion so far has centered on a filter 
which performs linear operations on the signal input to it. That is, it 
multiplies the Fourier components of that input by a given numerical 
factor and shifts their phase by a given angle. Both of these vary with the 
frequency of the component, but they are independent of the amplitude 
of the component. 

Some thought has been given by Bode and Shannon (Ref. 3) and others 
(Ref . 17) to the possibilities of nonlinear prediction, in which the operation 
on the signal would vary with the signal amplitude. At the expense of 
functional complications, this permits improvement in the accuracy of 
prediction under certain conditions. The problem has not been worked out 
to anything like the analytical detail devoted to linear prediction. 

6. NETWORK SYNTHESIS 

Introduction. Many of the problems discussed so far lead to a solution 
in the form of an electrical filter. Specification of the transfer properties 
of this filter comes out as the end product of the solution in terms of 
frequency as K(u), or of time as k(t). In an actual case a further step is 
necessary, namely the network synthesis. The filter or network must 
be built, and it needs specification in terms of components. 

This is an art with an extensive background and innumerable ramifica- 
tions. The scope of the present discussion is limited to electric networks, 
with some references for amplification. In data signal processing, on oc- 
casion the mechanical properties of equipment may affect signal propaga- 
tion. Some discussions of electromechanical elements have been given by 
Everitt and Anner, and by Graham (Ref. 18). 

The stage of analysis considered at this point bridges some of the steps 
from a theoretical toward a schematic design of the equipment. Some of 
the solutions which have been advanced in the present chapter, particularly 
to problems which are largely of prediction and exclude noise, are essen- 
tially formulas for analog computation (see Vol. 2, Analog Computers). 

Components of Electric Networks. The elements composing electric 
networks within the limited scope of this treatment consist of resistances, 
inductances, and capacitances. These elements are marked by various 
relations between voltage across them and current flow. 

The voltage may be set up as a function of time, as 

v(t) = Vq cos (cot + 0) 

or as a function of radian frequency, as 

V(u) = v e^. 

Here it is taken as a complex quantity. The current may be similarly 
expressed. 
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The relationship between the two, for a resistance, is 

V(u) = RI(u>), 

where R is the value of the resistance, (idealized as independent of fre- 
quency). For an inductance the relation is 

7(w) = ia>LI(w) 

and for a capacitance it is 

7(co) = I(u)/(iaC). 

The quantity 

Z(u) = 7(u)/J(w) 

is called the impedance, and its reciprocal 

7(w) = 1/Z(«) = I(u)/V(u) 

is called the admittance. 

The impedance (or admittance) can be expressed for any aggregation of 
interconnected elements ending in two terminals. As such it may be called 
the "driving point impedance" (or admittance) of that network at the 
specified pair of terminals. In an aggregation it is also possible to measure 
the voltage at one pair of terminals, and the current at another pair. 
Here the voltage to current ratio is called the transfer impedance between 
the two pairs of terminals, and the current to voltage ratio the transfer 
admittance. 

Specification in Terms of Transfer Response. The properties 
required in filters such as have been specified in Fig. 3, or in eqs. (16), (26), 
(43) and others, have been expressed as a transfer response. When a func- 
tion of frequency, it has been called K(u>), and when of time, k(t). Accord- 
ing to the circumstances of the particular equipment considered, the re- 
sponse as a function of frequency may be set up as a transfer impedance or a 
transfer admittance. It may also be a transfer ratio merely of voltages, 
or of currents. 

For simplicity, consideration here is limited to a case practical in vacuum 
tube circuitry, where the input to the filter is taken as a current, and the 
output from it a voltage. That is, K(<a) is identified with a transfer im- 
pedance, or Zr(w). 

Cauer's Method of Synthesis. In Fig. 15 a vacuum tube gives output 
I into the filter. The voltage 7 across the terminating resistance R drives 
a succeeding vacuum tube. The figure shows a generally practical case 
of the filter both starting and ending with a bridged capacitance. The 
filter itself therefore comprises an odd number of elements. If in any case 
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it is desired to omit one of the end capacitances it may be assumed to 
approach zero. 

For a filter of this type, the transfer admittance is a rational function 
of co with n poles and no zeros, and can be written as 



(67) 



Zr(co) = 



Zo 



(CO — COl) (CO — CO2) • • • (CO — Cd n ) 



Cauer (Ref . 19) has indicated a method for synthesizing the network from 
this function, which has been noted by Peless and Murakami (Ref. 20). 
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Fig. 15. Low-pass ladder filter. 

For this, Z is changed to a function of p = io, thus 

R 



(68) 



Zi(p) = 



b nV n + b n _ lV n - 1 +...+ ft lP +i 



The normalizing factor becomes R. 

Then the driving point impedance Z x {p) of Fig. 15 (excluding the 
terminating resistance R) is found by dividing the even part of the de- 
nominator by the odd part, and multiplying the whole by the normalizing 
factor R. 



(69) 



Zi(p) = R 



b n _ lV n - Y + b n _ 3 p n -" + • • • + b 2 p' + 1 



(70) 



b n p n + b n _ 2 p n - 2 +-.-+b lV 
Z is then expanded into a continuing fraction, as 

R 



Zi(p) = 



Q\V + 



02P + 



1 



9sP + 



1 

QnV 
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Z\ is found from the elements of Fig. 15 as 



(71) ZM = 



pC,+ 



pL 2 + 



2>C 3 + 



1 

pC n 



Thus 

Ci = gi/B, 

L 2 = g 2 R, 

(72) C 3 = g 3 /R, 



C n = g n /R- 



Illustration. The solution of the illustration expressed by eq. (43) 
represents an especially simple case. 

kR 

(73) Z T {p) = 



(e/Vl + e 2 )p + 1 



Here k represents a constant factor or gain adjustment to be set when 
lining up the equipment. 

(74) ZM = z-^— g, ' 

(e/Vl + e 2 )p 



(75) d = (e/Vl + e 2 )/B. 

Butterworth-Thomson Filters. A few general characteristics of the 
performance of niters like Fig. 15 may be noted. When a stepped wave 
signal, as indicated in Fig. 16a, is used as input, the output signal has the 
essential character of Fig. 16b. Where the input amplitude is current, and 
the output amplitude is voltage, this trace in Fig. 166 is called the indicial 
impedance of the filter. 

The trace is distinguished by a rise time, which measures the duration 
between crossings of 0.1 and 0.9 of the final amplitude. (Somewhat dif- 
ferent ranges are occasionally used.) The trace is also distinguished by 
an overshoot. This is measured as a per cent of the final amplitude. 
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In the design of such niters for general purposes (and in the absence of 
a specific formulation like the Wiener equations) it is usually desirable to 
conserve the shape of the input signal as much as feasible. This is obtained 
with a short rise time and low overshoot. 

It is similarly desirable to conserve frequency space. In terms of a 
response characteristic, such as illustrated in Fig. 9, the region where the 
response is large is called the passband. The region where it is small is 
called the elimination band. An intermediate region is called the rolloff 



Overshoot 



Time- 



(a) 




Fig. 16. Passage of step function signal through filter: (a) input, (b) output. 



band. Conserving frequency space consists in limiting the total band, 
comprising passband and rolloff band together. For a given passband it 
means limiting the ratio of the rolloff bandwidth to the passband width. 
Such conservation is significant in that it represents a system cost, in 
money, bulk of apparatus, and other factors, to maintain transmission 
over a wider frequency band than necessary. 

Within a given frequency space (passband plus rolloff band) rise time 
and overshoot conditions are mutually antagonistic. A plot of one versus 
the other, for some filter designs of the type of Fig. 15, is shown in Fig. 17. 
Here the rise time is normalized in a manner discussed below. 

A series of filter designs was presented by Butterworth (Ref. 21) charac- 
terized by a minimum of curvature of the response characteristic in the 
passband and called maximally flat amplitude. This is plotted in Fig. 
17 for m = (Butterworth). Here n has the same meaning as in Fig. 15. 
The design condition leads to generally short rise time, but fairly high 
overshoot. 

A similar series of designs was presented by Thomson (Ref. 22) charac- 
terized by a minimum of curvature in the phase characteristic and called 
maximally flat envelope delay. It is plotted in Fig. 17 for m = 1 (Thomson). 
This design condition leads to generally higher rise times, but lower over- 
shoots. 
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Fig. 17. Design curves, transitional Butterworth-Thomson niters. 

Transitional Butterworth-Thomson Filters. Peless and Murakami 
(Ref . 20) have prepared a series of designs intermediate between these two, 
by degrees indicated by the parameter m. Their rise time versus over- 
shoot performance is plotted in Fig. 17. 

In all these filter designs the ratio of rolloff bandwidth to the passband 
width tends to reduce as the number of elements is increased. This is why 
the better compromises between rise time and overshoot in Fig. 17 appear 
for the smaller numbers of elements. 

A rough measure of the utilization is given by the ratio of the frequency 
for which the drop in response is large (say 30 db, or an amplitude ratio of 
1 to 31.6) to the frequency for which the drop is small (say 3 db, or an 
amplitude ratio of 1 to 1.41). In the figure this quantity is called the 
cutoff flatness; it is plotted in dotted lines. Each line indicates a locus of 
compromises, between rise time and overshoot, for a given fixed degree of 
frequency band conservation. 
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The normalized rise time has been taken, in the Butterworth filters, to 
indicate the rise time of a filter whose response has dropped 3 db at a radian 
frequency of 1 radian per second (or a cyclic frequency of l/(2ir) cycles 
per second). The rise times for the other filters are for designs whose am- 
plitude response characteristics in the elimination band are asymptotic to 
those of the Butterworth filters. Where m is positive, the 3-db points of 
the niters come at lower frequencies than for the Butterworth filter. The 
exact amounts are indicated by co in Tables 1 to 4. 

Table 1. Two-Element Filters 



m 


-0.2 





0.2 


0.4 


0.6 


0.8 


1.0 


1.2 


CO 


1.054 


1 


0.949 


0.902 


0.859 


0.820 


0.786 


0.756 


C x uR 


























Ltfii/R 


0.747 


0.707 


0.673 


0.643 


0.618 


0.596 


0.577 


0.561 


CzcaR 


1.338 


1.414 


1.486 


1.554 


1.618 


1.677 


1.732 


1.782 






Table 2. Three-Element Filters 






m 


-0.2 





0.2 


0.4 


0.6 


0.8 


1.0 


1.2 


CO 


1.064 


1 


0.933 


0.868 


0.810 


0.756 


0.712 


0.671 


CiuR 


0.525 


0.500 


0.478 


0.458 


0.441 


0.425 


0.411 


0.398 


L^ji/R 


1.393 


1.333 


1.287 


1.252 


1.224 


1.201 


1.184 


1.168 



CzuR 1.368 1.500 1.624 1.743 1.854 1.958 2.055 2.148 



Table 3. Four-Element Filters 



m 

CO 

Cicofl 

L^/R 

C 3 coR 

L4O0/R 



-0.2 
1.064 

0.399 

1.127 
1.643 




1 

0.383 

.1.082 
1.577 



0.2 

0.924 



0.368 

1.043 

1.535 



0.4 

0.845 



0.354 

1.008 

1.508 



0.6 

0.774 



0.342 

0.978 

1.492 



0.8 

0.712 



0.330 

0.951 

1.484 



1.0 

0.659 



0.320 

0.928 

1.481 



1.2 

0.617 



0.311 

0.907 

1.482 



C b wR 1.353 1.531 1.698 1.856 2.004 2.143 2.273 2.395 
Table 4. Five-Element Filters 



m 


-0.2 





0.2 


0.4 


0.6 


0.8 


1.0 


1.2 


CO 


1.064 


1 


0.916 


0.824 


0.740 


0.671 


0.617 


0.572 


CiwR 


0.321 


0.309 


0.298 


0.288 


0.279 


0.270 


0.262 


0.255 


Ltfd/R 


0.928 


0.894 


0.864 


0.836 


0.811 


0.788 


0.767 


0.747 


C z wR 


1.435 


1.382 


1.338 


1.300 


1.269 


1.243 


1.221 


1.203 


L4C0/R 


1.770 


1.695 


1.656 


1.640 


1.638 


1.646 


1.659 


1.676 


Ct,uR 


1.323 


1.545 


1.751 


1.945 


2.125 


2.295 


2.453 


2.602 



Design Data. The information given above permits making a general 
compromise choice of filter for any given situation. The specific compro- 
mise choice depends upon what ultimate use is made of the signal. Where 
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the timing indication is important, the compromise would favor low rise 
time as against low overshoot. Where amplitude indication is important, 
the reverse holds. Where frequency conservation is important as compared 
with a more favorable rise time versus overshoot compromise, or where 
other specifications indicate need of sharpness in cutoff, enough elements 
to do this may be chosen. 

The element values, taken from the Peless-Murakami paper (Ref. 20), 
are listed in Tables 1 to 4. The values are normalized as for the rise times, 
and are also normalized to a termination resistance R. Thus the entry 
of 1 for CcoR means that C alone is 1 farad divided by 2x times the cyclic 
frequency of the Butterworth 3-db cutoff, and again divided by R. If 
that cutoff is 1000 cycles, and R is 1000 ohms, 

C = 1 X (2tt X 10 3 )- 1 X 10~ 3 farad 
(76) 

= 0.159 microfarad. 

Similarly the entry 1 for Lu/R means that L alone is 1 henry times R 
divided by 2ir times the cyclic frequency of the Butterworth 3-db cutoff. 
Again for a cutoff of 1000 cycles, and R of 1000 ohms. 

(77) L = 1 X (2tt X 10 3 ) -1 X 10 3 = 0.159 henry. 

The even element filters are all designed for C\ = 0. 

Tchebysheff-Darlington Filters. More general discussion of the 
synthesis of networks has been presented by Darlington (Ref. 23) and by 
Grossman (Ref. 24). The procedures there employed lead to the use of 
mathematical contributions of Tchebysheff, and the filters have been 
called Tchebysheff-Darlington filters. Essentially in the papers referred to, 
the design is specifically applied to filters in which tolerances are placed 
on a permissible variation in transmission over the passband, and on a 
required completeness of suppression in the elimination band. 
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1. INTRODUCTION AND SYMBOLS 

Basic Considerations. Data which are generated at a given point, 
either as a result of collecting original information or at the output of a 
computer as a result of the processing of other data, often have to be 
transmitted to some other point in order to be used for further data process- 
ing or remote control. Two basic parameters determine the extent of the 
undertaking which this transmission involves. 

1. The order of magnitude of the distance between the points of origina- 
tion and utilization. 

2. The nature of the data that are to be transmitted. This includes the 
information content of the data and the frequency band which is required 
to handle it in the transmission medium. For fundamental discussion on 
this point see Chap. 16. The treatment here analyzes principally the 
current practical art, in which the efficiency of utilization of the frequency 
band is much lower than ideal. 

The adaptability of the data signals to transmission over available 
facilities is a practical factor of great importance. There are extensive 
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systems of communication already set up, reaching over vast areas, which 
are in current commercial use. 

Transmission Distances Involved. The engineering effort required 
to set up a system of data transmission varies considerably with the 
distance involved. Some concrete illustrative gradations are : 

A few inches or feet 

One to a few hundred feet 

One to a few miles 

One to several hundred miles 

Several hundred to several thousand miles 

International or intercontinental facilities 

This discussion stresses particularly lengths in the middle regions, from 
a few miles to a thousand miles. 

Nature of Transmission Facilities Available 

These facilities need to meet certain requirements, discussed below. Of 
these the frequency band which they are capable of transmitting is para- 
mount, and is of chief concern here. 

Local Wiring. The band which this wiring will handle is indefinite, 
and it varies with the physical structure of the conductors and their in- 
ductive exposure to other electrical circuits. Bands have been handled 
from less than 100 cycles to television bands of a few megacycles. 

Telephone Facilities. These include all of the plant which has been 
developed for telephonic purposes, and hence they comprise a wide variety 
of facilities. They are sometimes nominally characterized as capable of a 
3-kc band. This full width band is not usable for data transmission, 
partly because some of the facilities cut off below this and partly because 
the lower frequency region, below 1000 cycles, is not likely to be effectively 
employed in the data transmission (Ref. 1). See Sect. 2 for more quantita- 
tive details regarding a usable band. 

There are telegraph facilities of narrower band, but since these are 
usually multiplexed on telephone facilities, they are not considered sepa- 
rately. 

Program Transmission. These circuits are commercially used for the 
interconnection of radio broadcast stations. They have frequency band- 
widths, in round numbers, of 3, 5, 8, and 15 kc (Ref. 2). As in the case 
of the telephone facility, the full band cannot be expected to be utilizable 
for data transmission. Also the commercial demand for the 8- and 15-kc 
bands is very low, so that there is at present a substantial network of only 
the 3- and 5-kc bands. 
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Television Transmission. An extensive network exists at present 
interconnecting television broadcast stations and studios and facilities for 
theater television (Ref. 3). The bandwidth of these facilities generally 
runs to a little over 4 Mc. However, on older coaxial cable facilities the 
bandwidth is only 2.7 Mc. Some experimental facilities of broader fre- 
quency band than 4 Mc have been furnished for short period tests, but 
not on a commercial basis. 

Other Wide Band Conductor Transmission. For economy, tele- 
phone facilities are frequently gathered together in more or less large groups. 
The combined signal for the entire group is then handled over a wire cir- 
cuit much as a single signal (Ref. 4). Groups of 48-kc bandwidth, and 
super-groups usually of 5 groups, or 240 kc, are handled in this manner. 
Also, on other types of system, bands of 16 kc are found. 

The use of these types of bands would, of course, require the development 
of arrangements for extending them from the terminals, at offices of the 
common carriers where they are located, to other premises. 

Carrier current facilities are also multiplexed by power companies on 
power lines. These are suitable for data transmission (Ref. 32). 

Radio Facilities. Radio facilities naturally present certain elements 
of flexibility in their use compared with facilities provided over conductors. 
The limits to this flexibility are, however, set by allocation problems and 
by the propagation characteristics of the frequency region used (Ref. 5). 
The frequency bandwidths used run from those for individual channels to 
large aggregations of multiplexed channels which may include television 
channels (Ref. 6). 

The utilizable bandwidth for the individual channels is not necessarily 
set by the adjacent allocations. It is often actually set by multipath echo 
effects. It tends to run from something under a telephone bandwidth 
(3 kc), up to the general order of magnitude of television channels (6 Mc). 

Radio channels that form part of a large aggregation, particularly those 
leased from common carriers, tend to run at telephone or television band- 
width, and differ little from similar circuits over conductor facilities. 
Similarly, group and super-group bands of intermediate width are trans- 
mitted, but the use of these again requires the development of arrange- 
ments for extending them from the terminals. 

Nature of the Data 

Data consist fundamentally of two types of information (Ref. 7). 

1. Choices among a group of possible conditions. A single datum, such 
as a room temperature, represents the single choice out of an established 
gamut. The total possible number of choices in that gamut depends both 
on the range of the gamut and on the precision of the indication within the 
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gamut. For example, a range of room temperatures may be established 
between 50° and 90° F, and the indication may be given to individual 
degrees. Then the datum represents one choice out of a possible 40, and 
it may be this which is to be transmitted. 

2. The timing of one or a series of events (Ref. 31). One might, for 
example, send the equivalent of a clock ticking from one geographical 
place to another, to assure the simultaneity of astronomical observations 
made at these places. 

In many cases in practice, both types of information may be needed. 
For example, in air traffic control, both the position of a given plane and the 
time at which it occupies that position, are needed. The position is in- 
dicated at the same time that it is occurring. Such a datum is said to be 
sent in real time, as distinguished from sending it much later on as a com- 
ponent in some abstract calculation. Data sent in real time are charac- 
terized by becoming "stale," i.e., of losing their value, if delayed too long 
in transmission. 

Continuous Analog Data. In the room temperature example cited 
above, the temperature may be represented by the position of the end of a 
mercury thread, or by the angular position of a shaft (dial thermometer), 
or again by the value of a given voltage. This position or voltage is not 
the actual temperature, but may be identified as analogous to it. Such 
data, where some different quantity varies proportionately (or according 
to some other appropriate law of variation) to the quantity desired, are 
called analog data. The demarcations between choices are not emphasized 
in the datum quantity, but they are important in a statement of the indica- 
tion. 

Where the analog relationship between the utilized data and the original 
quantity is not interrupted, the data are said to be continuous. 

Discontinuous Analog Data. It may suffice to have the temperature 
information once every 10 minutes instead of continuously. An analog 
quantity may be set up in which the relationship to the original quantity 
is interrupted when not needed. The results are called "discrete" or dis- 
continuous analog data. This may make it possible to interlace other data 
between the temperature readings. 

Multiple Speed Analog Data. In the case of a clock it has been found 
convenient to use the angular position of the shaft of the minute hand to 
identify one out of 60 choices, or one minute in the hour. For general use, 
however, a range of 12 hours is desirable. It is not convenient to use a 
pointer that can identify one out of 720 possible choices. The problem 
is solved by using two shafts, one geared to the other. The minute hand 
identifies one out of 60 choices. The hour hand identifies one out of 12 
choices, each of which corresponds to one group of the 60 choices of the 
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minute hand. The principle is sometimes extended by adding a third 
shaft and hand to read seconds. It is even further extended in conventional 
watthour and gas meters. All these are examples of "multiple speed" or 
"multiple shaft" analog data. 

Digital Data. The above process can be carried to its logical conclusion, 
where each shaft distinguishes only one out of two choices. In this extreme 
case the demarcations between choices are emphasized, and the choices 
would more usually be indicated by two-position members rather than 
by shafts. The choice may be considered as identified by a sequence of 
binary indications or binary digits and the data are called digital. Less 
extreme forms are sometimes used in which one out of three or more 
discrete choices are indicated by each digit. 

Digital information may be transmitted over a group of wires, by assign- 
ing one digit to each wire. This is known as parallel transmission. Or 
the various digits may be assigned to successive ordered pulses (or spacing 
intervals) on a single wire. This is called serial transmission, and the 
series of digits may be ordered in either direction. Examples. The digits 
indicating large values may come first (as in reading decimal digit Arabic 
numerals), or those indicating small values may come first (as in adding or 
multiplying operations with Arabic numbers) . 

Timing Data. This information can be indicated in a variety of ways. 
More usually the desired time is indicated by the wave front of an ap- 
propriate transition, say in voltage. 

Starting and Other Auxiliary Information. In the example just 
given above, where successive digits are transmitted serially, it is usually 
desirable to identify the start of the sequence by some auxiliary information. 
At other times the auxiliary information is in the nature of a pilot, reference 
or calibration datum against which the magnitude of the utilized data are 
compared before actual use. Other auxiliary information is sometimes 
needed for error checking or possible other purposes. 

Error Standards. It is not generally expected that data transmission 
will be completely perfect. For one or another reason, errors are caused. 
Thus in engineering a given system there is some need to give thought to 
what kind of error performance will be acceptable. 

In the case of analog data although the boundaries between successive 
choices are not emphasized, the spacing between the choices is important. 
This spacing is obtained from the precision which is found useful in the 
data. It is expected that this precision will be maintained in the trans- 
mission of the data. It is common to express the error expected or ex- 
perienced, in terms of its root-mean-square (rms) value (see Chap. 13). 
Occasionally a maximum error is noted, often say three times the rms 
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value. In a Gaussian distribution, errors larger than this occur with a 
frequency of about 1 part in 370. 

Timing errors are measured by a similar rms or maximum displacement, 
but in a timing variable rather than an amplitude parameter (Ref. 31). 

In the case of digital data an elementary measure of the error is the 
frequency of occurrence of errors in the binary digits in the data at the 
receiver. On occasion, a more sophisticated measure is desirable which 
takes account of the distribution of the errors in time. This is because 
in general, when one digit in a specific group of digits is in error, the use- 
fulness of the entire group is vitiated. Thus errors in close succession, in 
such cases, do not cause as much ultimate impairment as when they are 
more scattered. Measures of impairment in such cases are not easily 
established without some detailed knowledge of the entire scheme for 
setting up and using the data that are transmitted. 

Where special measures are incorporated into the signals for error check- 
ing, it is usually convenient to count the frequency of occurrence of both 
the detected and undetected errors. Of these, the first are apt to consti- 
tute only a minor impairment but the second are serious. Undetected 
errors are those not detected during the test, but obtained from some later 
comparison between the signals actually sent and those actually received. 



Symbols 

a relative echo amplitude 

D envelope delay 

/ cyclic frequency 

I wave amplitude 

k normalizing constant, equal to mean square value of I 

M matrix of resistances 

p(x) probability density of variable x 

R resistance, with subscripts for specific cases 

r amplitude ratio 

r(w) amplitude ratio at radian frequency « 

T pulse duration time 

t time variable 

V voltage 

v instantaneous signal voltage 

pulse separation time (front edge to front edge) 

X wavelength of ripple along cyclic frequency scale ( = A/) 

r echo delay 

0, 4>(w) phase shift at radian frequency w 

w radian frequency ( = 2ir/) 
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2. FORMATION AND USE OF THE ELECTRICAL SIGNAL 

Encoding and Decoding 

The first step in the preparation of a signal for transmission consists in 
expressing the variable that is intended for transmission into some sort of 
code that can be used to form the electrical configuration. 

Analog Data. There is not very much latitude for coding such data, 
aside from transferring from one type of physical quantity into another. 
Thus a temperature or a distance may be transformed into a shaft rotation 
or a voltage. The principal modification that can be introduced is the 
insertion of some sort of nonlinear relation between the one quantity and 
the other. 

Digital Codes. A simple code into which an analog quantity may be 
converted is the binary digital code (see Chap. 16). A diagram of the 8 
choices for a 3 binary digit code is illustrated in Fig. 1. The dark areas 
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Fig. 1. Diagram of 3-digit binary code selections. 

indicate, say, voltage (or current) "on," and the white areas, voltage (or 
current) "off." They are termed respectively marking and spacing. A 
variation of this code sometimes used to simplify an encoding mechanism 
is the reflected binary or Gray code (Ref. 8). This is shown in Fig. 2. The 
simplification in mechanism comes 
essentially because the change from 
any given choice to the next adjacent 
choice involves the change of only 
one binary digit. Other variations 
of this simple code type have been 
devised. One such is a coding to in- 
clude negative values of the encoded 
quantity (Ref. 9). 

More complicated codes have been devised in which present code designa- 
tion is a function also of past history of the value being encoded, or of 
more than one variable (Ref. 10). 

These complications are conceived principally to condense the informa- 
tion to be transmitted into the most compact code possible. They do 
involve an increase in cost of equipment, and a loss in time at both en- 




Fig. 2. Diagram of 3-digit reflected 
binary (Gray) code selections. 
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coder and decoder that may be important where the transmission operates 
in real time (see Sect. 1, Nature of the Data). Some economic study has 
been made of such points (Ref. 11). See also Chap. 16. 

Processes of Digital Encoding. Only some elementary principles can 
be mentioned here (Ref. 9). More details of these processes are given in 
Vol. 2, Chap. 20. 

1. A basic method of encoding consists in laying out the analog input 
along one dimension of a two-dimensional code matrix and reading the 
coded output along the other dimension. This can be utilized for any 
arbitrary code. As an illustration the diagrams of Figs. 1 and 2 may 
represent plates, with holes punched through the shaded squares, in a 
cathode ray tube (Ref. 12). The electron beam is deflected along the 
horizontal coordinate by analog voltage input. A subsequent vertical 
deflection then gives the coded signal, in serial form, on an electrode beyond 
the plate. The beam goes through the punched holes, but is stopped where 
no holes exist. 

2. A second basic method consists in encoding the analog quantity 
first into a unit-counting code. For each value of the analog quantity to 
be transmitted a counting mechanism counts and cumulates unit incre- 
ments up to a value nearest to the input quantity. A unit-counting code 
is not efficient for transmission since the number of binary digits sent is 
large. It can be converted into a binary digital code by successive scale 
of two counting dividers (see Vol. 2, Chap. 20). If other codes are desired 
a further conversion can be made. 

3. A third basic method uses the general principle that any decoder 
may be used for encoding by associating it with an appropriate inverse 
feedback path. An arbitrary code indication is set up, say the last previous 
transmission. This is decoded, and the result is compared with the in- 
put. The inverse path mechanism uses the error to step the code in the 
direction to reduce the error. The stepping mechanism continues until the 
error is less than the smallest choice interval. 

Coding mechanisms in which the present output depends on more than 
the single present input exhibit a greater variety of types and will not 
be discussed here. 

Processes of Digital Decoding and Smoothing. The decoder in 
general has two broad functions. 

1. Decoding proper is to convert the digital indication into an analog 
indication. In nearly all cases this appears as an individual analog in- 
dication when the code is received. In some cases this is the only function 
needed. 

2. To hold or store and possibly smooth the analog indications are needed 
where individual indications are required at more frequent intervals than 
the code permits, or where continuous analog indications are required. 
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The decoding function proper may be classified by types of mechanism, 
as for the encoder. For the moment these are limited to the case where 
a single input leads to a single output. 

(a) In a basic type of decoder the choices indicated by the respective 
binary digits lead to the single element of an arbitrarily prearranged matrix. 
This element translates to its prearranged analog output. 




Fig. 3. Relay matrix for 3-digit code. 

An example of this is shown in Fig. 3 in terms of relays. The respective 
digits operate relays 1, 2, and 3. Any given choice leads to some resistance 
of the matrix M. These are chosen in advance to yield the desired 
analog voltage V at the output, for the given choice. 

(b) A variation of this is applicable to codes where the successively 
ordered digits contribute proportioned weights to a cumulation of the 
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Fig. 4. Relay matrix for 3-digit binary code. 



analog total. This occurs in the binary digital code. An example is shown 
in Fig. 4. The successively ordered digits choose respective resistances 
Ri, R2, and R 3 . These are proportioned to cumulate currents, in the 
progressive ratios of 4, 2, and 1, in the output resistance, which must be 
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low compared to the R's to keep the contributions independent. The 
output voltage V gives the analog for the binary digital choice. 

(c) If instead of current contributions, successive pulse counts are 
cumulated, the process leads to a translation from a binary digital to a 
unit-counting code. This can then translate further to the final analog 
quantity. Such an arrangement is the inverse of the second basic encoding 
process. 

(d) Finally an encoder may be inserted in an inverse feedback path 
for conversion into a decoder. An arbitrary analog output is set up, say 
the last value just previously decoded. This is encoded, and the code 
indication is compared with the present input code. The inverse feedback 
path uses the difference to change the analog value in the direction to 
reduce the difference. Several steppings of the code may be needed until 
identity is secured. This basic method is not easily applicable to arbitrary 
codes. 

Holding the analog signal requires some form of temporary storage 
(Ref. 13). Where the error objective calls for a more accurate interpolation 
between the discrete values, still more equipment is needed. The process 
has been here called interpolation, but it is clear that after one discrete 
value has been obtained and before the next is available, the process really 
required is extrapolation or prediction. 

The principles are described in Chap. 17 for the optimizing properties 
required in the above processes. More than just an electrical filter may 
be needed, because of the discrete character of the values. 

Where the data are such that the best correlation occurs between succes- 
sive values of the analog variable, a mere holding, or zero order prediction, 
is optimum. Where, as is quite possible in practice, a good correlation holds 
between successive rates of change (or velocities), a first order predictor 
is better. This predicts from the velocity as derived from past data. Where 
a good correlation held on the accelerations, a second order predictor would 
be called for. 

This second function of the decoder may be used by itself in cases where 
the data were merely sampled as discrete analog data, and not digitally 
encoded. 

Error Detection Codes. It is possible to introduce deliberate redun- 
dance into the code used in the data transmission path. This establishes 
auxiliary relationships. At the receiver a test may be made for these 
auxiliary relationships. When they are found missing, the fact is an 
indication of error in the transmission. 

A simple form of this redundance is the parity check (Ref. 14). For 
this the message is divided into successive groups of binary digits, and an 
extra digit is provided at the end of each group for the redundant informa- 
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tion. The number of marks in the group is noted as being even or odd. 
If even, the added digit is made marking; if odd it is made spacing, to 
make the total always odd. Hence the reception of a total even number 
of marks indicates an error. Undetected errors can exist when two errors 
conspire to maintain the total odd. The system can also be arranged 
to make the correct total always even. 

Another example is the 2 out of 5 code. Here 5 binary digits are always 
disposable for the signal, and 2 of these are always made marking. This 
gives 10 combinations, which is very handy for translation from and to a 
decimal digit code. When the receiver receives any other than 2 marks, 
it indicates an error. Two errors can combine here also to evade detection. 
The code may also be used with 3 marks out of the 5 binary digits. 

The redundance may be increased to the point where the specific digit 
in error may itself be located in the signal, and therefore corrected. This 
is an error-correcting code. Some combinations of errors exist that can be 
detected by this, but not corrected. Even rarer combinations are possible 
which evade detection. 

Modulation and Multiplexing Methods 

Several steps must be considered in these processes. 

Baseband Signal. The information which is to be transmitted from 
one point to the other eventually appears in the form of an electric ampli- 
tude (say a voltage or a current) before it is propagated over the trans- 
mission medium. In a continuous analog system it appears, say, as a 
continuously varying d-c voltage. In a discrete analog system it appears 
as a succession of pulses of varying voltage amplitudes. In a binary 
digital system it appears as a succession of pulses each individually of 
either marking or spacing voltage amplitude. 

The signal in this form is called a baseband signal. It has a frequency 
spectral distribution of power which goes down to and includes zero fre- 
quency (or d-c). Its amplitude distribution depends upon the shaping 
of the individual pulses or shape factor (where pulses are involved) and upon 
the sequence of amplitudes which codifies the information or discrimination 
factor (Ref. 15). 

Nyquist has shown (Ref. 15) that the complex amplitude at each frequency 
is equal to the product of these two factors, each of which is complex. In a 
code that gives sufficient randomness to the signal, and that permits 
positive and negative voltage values, with an average of zero, the long time 
average of the power distribution in the discrimination factor is flat over 
the frequency range. In such a system, therefore, the signal power distribu- 
tion is equal to that for the shape factor, or for a single pulse (aside from a 
normalizing factor) . 
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For an idealized pulse with rectangular sides, such as shown at A in 
Fig. 5, the frequency band is infinite, as illustrated by the full line. How- 
ever, most of the power is located below the frequency 1/T, where T 
is the pulse duration. 

Practical pulses are in general rounded somewhat as shown at B of 
Fig. 5. For these, all but a negligible proportion of the power is located 
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Fig. 5. Power spectra of various pulses. 

below frequency 1/T, as illustrated by the dotted line. The pulse form 
at C indicates that obtained from the full line spectrum, cut off to zero 
above frequency 1/T. 

Nominal Effective Band. Nyquist has further shown (Refs. 10 and 
15) that the minimum frequency band required to transmit independent 
amplitudes for each pulse, where the pulse separation is 0, is 1/(29). This 
may be called a nominal effective band. In practice a somewhat larger band 
is generally used. If the successive pulses are set up edge to edge (that is, 
8 made equal to T), then in the full line of Fig. 5, the nominal effective 
band reaches from to l/(27 1 ). The band actually used in practice usually 
reaches to 1/T, or twice as far. The part of the band between 1/(2T) and 
1/T transmits a portion of the signal that has low power, and may be 
designated as rolloff band. 

Occasionally a narrower rolloff band than that reaching to 1/T may be 
used. This results in oscillatory transients or "ringing" before and after 
each pulse of the signal. 

Where short pulses are transmitted at infrequent intervals, 8^> T. In 
such cases a wider frequency band is used than necessary for the informa- 



DATA TRANSMISSION 



18-13 



tion, and 1/T » 1/0. Additional pulses from other channels of informa- 
tion can be interlaced in between, to use the frequency space more fully. 
It is found in Sect. 3, Tolerances, that this leads to the need for high 
fidelity in the transmission. 

Amplitude Modulation. Except for direct wire transmission over 
short distances it is not usually practicable to transmit a baseband signal 
of the spectral distribution illustrated in Fig. 5. This is because it involves 
transmission all the way down to and including d-c. 




Frequency 
Fig. 6. Spectra of baseband and carrier pulses. 



A simple procedure to avoid the need of d-c is to use the baseband 
signal to form the envelope of a carrier wave, as shown at B in Fig. 6. 
Here the spectrum becomes that of the carrier frequency and two symmet- 
rical sidebands. Each of these has the same shape as the baseband. The 
baseband or envelope signal is recovered at the receiver usually by the 
use of a rectifier. 

Certain precautions are needed when using a carrier signal in this way. 
Interferences are likely to develop between the lower sideband and the 
baseband, if the carrier is placed at such a low frequency that they overlap. 
Each of these interferences can, if needed, be reduced at the source. The 
more usual procedure, however, is to allocate the spectrum to avoid such 
an overlap. 

Another point to be noted is that over certain types of telephone facili- 
ties, in which another carrier wave is used for the transmission, the data 
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signal carrier may not be reproduced at its exact frequency. The received 
signal may be displaced up to two cycles per second from that transmitted 
(in some older facilities this may be some 20 or 30 cycles per second). Thus 
the system cannot be designed in such fashion that the reproduction of 
this exact frequency is critical. For example, the carrier frequency cannot 
be depended upon for use as a synchronizing frequency. 

Vestigial Sideband Transmission. The information carried in one 
sideband is duplicated in the other. Thus only one of the two is necessary 
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Fig. 7. Vestigial sideband spectrum. 

for transmission, and a saving in frequency space required in the trans- 
mission medium is achieved by suppressing the other sideband. How- 
ever, cutting off a band sharply at the carrier is difficult in data trans- 
mission, where the power spectrum contains frequencies close to and in- 
cluding the carrier. To solve this Nyquist (Ref . 15) has indicated a sloping 
cutoff as shown in Fig. 7. This retains a "vestige" of the suppressed band, 
and is, therefore, called vestigial sideband transmission. The cutoff through 
the carrier region introduces interfering spurious components to the signal. 
These are called "quadrature" components because in their interfering 
effect they add in quadrature to the undistorted signal. The interference 
usually causes an impairment in signal-to-noise ratio, which is discussed 
more fully in Sect. 3, Quadrature Component in Vestigial Sideband Trans- 
mission. 

When all the precautions which have been discussed are allowed for, 
it is found that with double sideband modulation a signaling speed of about 
650 signal elements (each of duration T = 1.54 milliseconds in Figs. 5 
and 6) per second may be transmitted over a substantial proportion of tele- 
phone circuits (Ref. 16). 

With vestigial sideband transmission about 1600 signal elements per second 
have been transmitted over selected and suitably treated telephone circuits 
(Ref. 17). This is slightly more than double that with double sideband. 
The increase comes mostly from the use of the vestigial band, but in part 
from selection and treatment of circuits. 
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Frequency Modulation. Characteristics of the carrier wave other 
than its envelope amplitude may be varied in accordance with the base- 
band signal. A common example is the variation of its instantaneous 
frequency. This can have certain advantages, for example, when transmit- 
ting over a medium whose amplitude response at the receiver varies from 
instant to instant. A more detailed analysis also shows that transmission 
by frequency modulation is less subject to impairment from noise than by 
amplitude modulation (Ref. 18). 

Other Methods. Still other characteristics of the carrier wave may be 
varied to indicate the signal (Ref. 19). Phase modulation may be used. 
Or the data signal (whether itself constituted of pulses or not) may be 
transmitted over a medium that uses a pulse code form of modulation 
(Ref. 20). In this case the instantaneous amplitudes of the data signal 
are reproduced by a secondary pulse code. However, the requirements 
for this mode of transmission have not as yet been worked out. The range 
of different possibilities is very great. 

Frequency Division Multiplex. In Fig. 6 a second carrier with its 
sidebands may be placed at a higher frequency than B, and transmit an 




Frequency 
Fig. 8. Carrier signal spectra, frequency discrimination. 



independent signal. More carriers can, of course, continue to be added, 
as suggested in Fig. 8. The limit is the frequency band available on the 
transmission system. This is known as multiplexing by frequency division. 

A consequence of this form of multiplexing lies in modulation products 
which are generated between pairs (and larger groups) of the simultaneously 
operating channels. These products can cause interference into the chan- 
nels in which they fall. The modulation arises from nonlinearity in the 
transmission process, at a possible variety of points according to the 
details of the facility used. Engineering precautions are needed to keep 
the interference down to an acceptable level. 

Time Division Multiplex. Where a signal uses a basic pulse which is 
repeated at much longer intervals than its own duration, other independent 



18-16 



INFORMATION THEORY AND TRANSMISSION 



signals may use pulses which are inserted intermediate between these. A 
scheme of five such channels is indicated in Fig. 9. The limit is, of course, 
fixed by the relative durations of the individual pulses, the spacing interval 
between pulses of the same channel and the guard space which is required 
between pulses of one channel and of its nearest neighbors to prevent 
mutual interference; 

The choice of whether frequency or time division multiplexing is prefer- 
able in a given case depends upon the nature of the transmission impair- 
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Time 
Fig. 9. Pulse signal profiles, time discrimination. 

ments to be expected and upon the relative costs. Both methods are in 
extensive use. 

Other Forms of Multiplexing. In a generalized study of multiplexing 
(Ref. 21) it is found that n independent channels can be multiplexed on a 
given signal through the superposition of n mutually orthogonal functions. 
The arrangements discussed above represent two possible solutions, but 
there are many others. A single example is the use of independent ampli- 
tude modulation channels on the sine and cosine waves of a carrier. 

Although these other orthogonal function solutions offer possibilities in 
the art, they have not at the present time received as much design effort 
on actual embodiments as have the frequency division or the time division 
multiplexing. 

Auxiliary Signals 

It was noted in Sect. 1, Nature of the Data, that when the data are 
presented in certain ways it is desirable to include some starting or other 
auxiliary information to mark out specific blocks of the information or to 
give other reference conditions. This auxiliary information needs to be 
distinguished in some way from the primary information. It comes 
regularly in the organization of the transmission, so that while the system 
is in normal operation the distinction need not be particularly conspicuous. 
However, for one cause or another the transmission may occasionally be 
interrupted. When this occurs, reestablishment is likely to be quicker, the 
more distinctive the auxiliary signals are. 
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Multiplexing the auxiliary signal with the principal signal may be done 
in a large variety of ways. A simple form is used in the standard tele- 
typewriter. Here the distinction is secured by setting up a pattern of 
marking and spacing in the binary signal that is not duplicated in any 
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Fig. 10. Stop and start pulses in teletypewriter signal. 

portion of any character. As indicated in Fig. 10, this pattern consists 
of a marking stop signal that in duration is equal to or greater than 1.4 
signal elements, followed by a spacing start signal of one signal element 
duration. At a and b are shown stop signal elements of minimum duration, 
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Fig. 11. Double sideband signal with auxiliary word start channel. 

at c one of longer than minimum duration. There is, of course, some possi- 
bility that the excess over the minimum duration would bring the stop 
signal to 2 signal elements. This could then be duplicated in portions of 
characters, which could then be confused with the stop-start pattern. 



■Start 




Fig. 12. Use of amplitude discrimination for word start channel. 

This does occur on occasion, and the return to normal operation takes 
several characters. 

At the other extreme, distinctiveness in the auxiliary signal is obtained 
by the use of an extra, narrow band carrier channel for it. One example 
of this is illustrated in Fig. 11. Another method is to use amplitude 
discrimination (Ref. 17). This is illustrated by the signal of Fig. 12. 
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With this arrangement the amplitude range permitted for the signal is 
reduced in comparison with the power capacity of the transmission 
medium, since the maximum capacity is used for the auxiliary signal. 
Thus the effective signal-to-noise ratio is less than it could be if the principal 
signal utilized the full capacity. 

There are numerous other methods of introducing the auxiliary signals. 

3. TRANSMISSION IMPAIRMENT 

Electric circuits do not reproduce signals with complete fidelity. A 
basic element of the engineering of a system consists, therefore, in establish- 
ing tolerances on the permissible impairment of the signal consistent with 
acceptable performance. 

The influence of limitation of the frequency bandwidth has already been 
discussed in Sect. 2. 

Noise 

No electric communication circuit is ever free from varying currents and 
voltages which are uncorrelated with the transmitted signal (except 
possibly in some statistical manner) and which tend to be confused with 
it at the receiver. These erratic waves have been perceived and studied 
in telephony. They have there been named noise because they end in 
audible noise in the receiver. The term has, however, been generally 
extended in the art to cover the effect in other types of communication. 

Since noise is unpredictable in detail, it has in general to be dealt with 
in a statistical manner. Extensive study has been made of its statistical 
and other properties (Ref. 22). The discussion here is confined to a simple 
exposition of what one can expect in signal transmission media. 

Single Frequency Noise. The noise wave may consist of a sustained 
single frequency. On the time scale this is a simple harmonic variation in 
voltage or current. On the frequency scale it is a single line spectrum. 

Single Impulse Noise. On the other hand, the wave may consist of a 
sharp impulse at a given time. On the frequency scale it consists of a 
density of components which is uniform in amplitude out to some fre- 
quency beyond which it drops and approaches zero. This frequency de- 
pends upon the duration of the impulse. For an infinitesimal duration 
the frequency is infinity. The phases of the components are closely cor- 
related. 

Cumulation to Gaussian Noise. It is clear that the single frequency 
and the single impulse represent opposite extreme types of noise. In 
practice one can encounter a cumulation of a number of different single 
frequencies, each of different amplitude and phase. One can also encounter 
a cumulation of different single impulses, each of different amplitude and 
timing. 
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Each of these cumulations, as it becomes more extensive, and with 
sufficient randomness in its components, approaches "white" Gaussian 
noise (Ref. 23). White Gaussian noise may be defined in simple terms as 
random noise which has a Gaussian distribution of amplitude in the time 
domain and uniform distribution of power in the frequency domain. (To 
keep the total power finite, the uniformity need extend only somewhat 
beyond the frequency range under consideration for the signal.) The 
term "white" stems from its analogy to Rayleigh-Jeans radiation in optics 
(Ref. 24). This is somewhat of a misnomer, however, in that this radiation 
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Fig. 13. Normalized Gaussian and Rayleigh noise distributions. 



has a uniform distribution of power in the frequency spectrum. A white 
radiation used by colorimetrists has uniform power distribution in the 
wavelength spectrum. Thus Rayleigh-Jeans radiation, which has more 
power than this in the blues and less in the reds, is not white but really blue. 
The Gaussian distribution in a normalized form is expressed by the 
equation 

/ I \ dl _ dl . 

(1) V Wk) Vk ~ \S 6 



-PI 2k 



This is the probability that the instantaneous amplitude lies between 
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I/\/k and (7 + dI)/-\/k, where k is the normalizing constant, equal to 
the mean square value of I. 

A plot of this normalized distribution is illustrated in the full line of 
Fig. 13. The normalization consists in referring to the amplitude I in 
terms of its ratio to the rms value -\/k. 

Noise encountered in practice is rarely apt to be exactly any one of the 
three types which have been described. These are, however, much used 
as idealizations for mathematical and engineering purposes. 

Non-White and Non-Gaussian Noise. On occasions where the 
noise actually encountered is sufficiently different from the idealization to 
influence a conclusion it is necessary to deal with the non-white and non- 
Gaussian noise as such. 

The deviation of noise from the idealized white Gaussian may be charac- 
terized in a number of different ways. It may be characterized by size, 
as large or small; toward single frequency idealization or toward impulse 
idealization; by variation in spectral distribution or in distribution of 
amplitudes as a function of time; or other characteristics. 

One frequently used variant is noise obtained from Gaussian noise by 
envelope rectification. This follows a Rayleigh distribution of amplitudes 
(Ref . 23) . The Rayleigh distribution in normalized form is expressed by 
the equation 

/ i \ dl 21 dl T2I1 

(2) Avi)vi = — e ~" k> I>0 

= 0, I < 0, 

where the symbols have the same meaning as in eq. (1). 

A plot of this distribution, in the normalized form, is illustrated in the 
dotted lines of Fig. 13. 

A second frequently used variant is filtered white Gaussian noise. This 
modifies the power spectrum of the noise. A special case occurs when the 
filter has a passband which becomes narrow compared to the spectrum 
of the signal which is being disturbed. In this case the noise approaches 
single frequency noise. 

A third variant is the impulsive noise encountered in communications 
circuits. This is cumulated from single impulses of the type which have 
been considered. However, the number cumulated is small enough and 
not sufficiently random in amplitude or time of occurrence, so that the 
distribution of amplitudes (including zero amplitude, which is important) 
is not Gaussian. This can occur for example in telephone circuits exposed 
to some forms of dial-switching equipment or to static. It is necessary 
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for clpse engineering in such cases to determine the exact distribution of 
amplitudes, and sometimes of timing instants. 

Influence of Noise on Error. The effect of noise, of course, is that it 
changes the received wave and tends to cause the signal for one set of 
data to be confused with that for another. 

Case 1. Analog Data. The error may vary continuously from zero to 
large amounts. One simple method of expressing the error is in terms of 
its rms value. There are other methods (see Chap. 17), but they usually 
lead to more complicated techniques of engineering. The optimum noise 
performance may be obtained in a given system when both signal and 
noise are filtered through the optimum filter (see Chap. 17). 




Time >- 

Fig. 14. Effect of noise in causing errors in data pulse signals. 

Case 2. Digital Data. The received signal wave may vary over a 
range without any misinterpretation resulting (Ref. 25). When the de- 
parture exceeds this range, however, a marking signal may be misinter- 
preted as a space, or a space as a mark. 

A simple illustration of this is shown in the baseband signal of Fig. 14. 
At a is the assumed signal, consisting of a space, a mark, two spaces, two 
marks, and a space. The interpretation of marking or spacing is assumed 
to be made according to whether the wave amplitude falls above or below 
the critical value b, at sampling instants which are designated on the 
line b. The effect of a few noise pulses are indicated by dotted lines on a. 
In the simple baseband system shown, the critical level b is at half the 
marking level, or 6 db below marking. Thus the critical signal-to-noise 
ratio, in terms of marking level to noise peaks, is 6 db. For a higher signal- 
to-noise peak ratio, no errors in transmission are caused by the noise. For 
a lower ratio, errors appear. 

Because of the erratic nature of noise, this is not always a convenient 
specification. If the noise has, say, a Gaussian distribution, no matter 
what the critical level b is, there is some finite probability that it will be 
exceeded (with the appropriate polarity) and cause an error. The engineer- 
ing of the system then consists in first setting an acceptable error perform- 
ance (see Sect. 1, Error Standards). This sets the acceptable probability 
for the existence of noise pulses of a given polarity 6 db below marking 
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level. In Fig. 13 one can tell, for Gaussian noise, how far above rms a 
level must be to occur with any given probability. This indicates how 
far below the critical level b the rms of the noise must be set. If that 
ratio is translated into db, then by adding 6 db one obtains the figure which 
must be specified for the signal- (marking level) to-noise (rms) ratio for the 
system, in order to meet the desired error performance. 
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Fig. 15. Error probabilities from Gaussian and Rayleigh noise distributions. 



For convenience this ratio in decibels has been plotted as the solid line 
in Fig. 15. A dotted line is shown for the case of noise having a Rayleigh 
distribution. 

If the noise is of the impulsive type it is usually impractical to specify 
it in terms of its rms value, as a ratio to the signal. In such cases it is 
apt to be expeditious to make measurements on the noise itself, to deter- 
mine the amplitude of its peaks at the error frequency which has been set. 
Then the marking amplitude can be set 6 db above this. 

The noise margin of 6 db which has been discussed here is a basic figure. 
If the signal has three levels to be recognized, as in Fig. 12, the figure 
has to be increased. It will also be found (see below) that other margins 
need to be added to allow for other impairing effects on the signal. 

Echoes and Equalization 

Echoes and Transfer Functions. In general, no transmission medium 
reproduces a sent signal wave shape exactly in all its characteristics. The 
departures from exact faithfulness can be considered from two points of 
view, sometimes one being more convenient and sometimes the other. 
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According to the first point of view the departures may be considered 
as a succession of more or less delayed echoes of the original wave. Some 
of the echoes are of the same polarity as the original wave, and some of the 
opposite polarity. 

According to the second point of view the signal and its transmitted 
reproduction may be analyzed into their Fourier transforms. Each Fourier 
component of the reproduction is obtainable from that of the original 
by multiplication by an amplitude response factor and displacement by a 
phase shift. The factor and phase shift vary from frequency to frequency 
over the spectrum of the signals. Since the Fourier transform is unique, 
the complete description in a linear system of a given case according to 
the one viewpoint can also be matched exactly by a complete description 
according to the other viewpoint. Whichever one is used is then simply a 
matter of engineering convenience. 

Practical experience indicates that the echo treatment leads quickly to 
equalizer designs. This is because it forms an immediate bridge to func- 
tions of frequency, and equalizer designs are simple in such terms. The 
design of filters and equalizers from transfer functions of time is usually 
far more cumbersome. 

Equalization. In practical transmission systems these distortions are 
usually reduced by what is called equalization. A network is placed in 
tandem with the system which again multiplies all the Fourier components, 
each by an amplitude response factor, and displaces each by a phase shift. 
The response factor of the network is designed to vary in the inverse way 
from that of the system, so that the products of the two are as nearly as 
possible constant over the frequency spectrum. For this reason the net- 
work is called an equalizer and the process called equalization. The phase 
shifts are designed to add together to a total phase shift which is pro- 
portional to frequency. Since the perceptive mechanism of the ear is not 
very responsive to phase shifts, the equalization of telephone circuits has 
generally concerned itself almost exclusively with an equalization of the 
amplitude response factor. This unconcern with the phase correction is 
occasionally of importance in the use of telephone facilities for data trans- 
mission. It sometimes requires the insertion of phase correcting networks 
to supplement the amplitude correction already existing for the telephone 
use. 

It is not usually possible or economically feasible in practice to correct 
a system exactly. The considerations which are given below can apply 
equally well to a residual departure, left after such correction has been 
applied as is practical, or to an uncorrected facility. 

Impairment of Noise Margin. It is clear that an echo partakes of 
one property in common with noise, i.e., it changes the received wave and 
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tends to cause the signal for one set of data to be confused with that for 
another. Where a given deviation has previously been set as acceptable, 
the echo uses up some of this possibility for deviation and leaves less of it 
as an allowance for the noise. In this sense, therefore, it impairs the noise 
margin of the system. Where, as was noted above, a margin of 6 db was 
necessary for the marking signal level over individual noise peaks to just 
avoid potential error, a greater margin is needed in the presence of echo. 

The amount of this excess margin which can be allotted to echo, in any 
given case, is a matter of engineering judgment. It depends upon the 
relative costs of reducing the echo and the alternative of reducing the 
noise. One may say that, in general, an increase of 1 db in margin is small, 
and that as severe a limitation as this on the echo is economical where the 
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Fig. 16. Close-in and remote echoes in signal. 

echo is fairly easy to reduce. At the other extreme one may say that an 
increase of 10 db in margin is fairly large. It is likely to be economical 
only where it is quite difficult to reduce the echo, or where the noise ex- 
pected is very low. In the engineering of data systems it is convenient to 
consider steps respectively of 1, 3, 6 and 10 db in the noise margin impair- 
ment. 

The amplitude of echo that can cause a given impairment depends upon 
how much it is delayed with respect to the original signal. As an illustra- 
tion the signal A in Fig. 16 may be followed by a comparatively long 
delayed echo at B. Here the impairment depends upon the echo amplitude, 
and it does not vary much with small changes in the echo delay. 

The signal may also be followed by a closely spaced echo as at C. The 
sum of signal and echo is shown at D. Here the major effect of the echo 
is to change somewhat the wave shape of the signal, but mostly it changes 
signal amplitude. A substantial part of the effect of the echo consists 
merely in changing the effective loss of the transmission facility somewhat. 
This part of the effect can be compensated for by an adjustment of receiving 
gain. The impairment from a closely following echo of a given amplitude 
is less than from a long-delayed echo of the same amplitude. Also the 
impairment can be expected to vary rather rapidly with delay, for the 
short delays. 

Relationship between Echoes and Equalization. This relationship, 
suggested above, may be examined quantitatively (Refs. 7 and 26). 
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Consider a single Fourier component of the signal voltage, of frequency 

co/2r 

(3) v = cos wt. 

When this is transmitted over a system that generates an echo of relative 
amplitude a and relative delay r, it becomes 

(4) v = cos wt + a cos u(t — r) 

= cos at + a cos at cos cor + a sin at sin cor, 

(5) « = (1 + a cos cor) cos cot -f- a sin co£ sin cor. 
If the overall transmission is designated as 

(6) v = r(co) cos [cot — 0(co)], 

then r(co) represents the overall amplitude ratio and $(co) the overall phase 
shift. These quantities may be computed from eq. (5), namely as: 

(7) r(co) = [(1 + a cos cor) 2 + (a sin cor) 2 ]^, 

(8) ~ 1 + a cos cor, when a is small; 

, a sin cor 

(9) tf>(co) = tan -1 . 

1 + a cos cor 

(10) ~ a sin cor, when a is small. 

Thus a given echo in a description in terms of time corresponds approxi- 
mately to a ripple in the amplitude response and in the phase, in a descrip- 
tion in terms of frequency. The amplitude of the ripple, in nepers or radians, 
is equal to the relative amplitude of the echo. The wavelength of the ripple 
along the frequency scale (X = A/ = Aco/27r) is equal to the reciprocal of 
the delay r. 

Acor = 27r, A/r = 1. 

(11) 

X = A/ = 1/r. 

Tolerances. A given amplitude allowance for noise and echo together 
is taken as unity, and the amount allocated to noise alone is allocated at 
1 to 10 db below this. This noise allocation bears a ratio of r (<1) to 
the total. The remainder, or 1 — r, is allocated to the echo. Figures 
are given in the first two columns of Table 1 to indicate the numerical 
values. 
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Table 1. 


Echo and 


Ripple Tolerances 




Impair- 


r = Ampli- 


Echo Tolerance Ripple Tolerance 


ment, db 


tude Ratio 


(1 - r)/2 


db (1 - r)/2, db 


degrees 


1 


0.89 


0.054 


25 ±0.47 


±3.1 


3 


0.71 


0.146 


17 ±1.3 


±8.4 


6 


0.50 


0.25 


12 ±2.2 


±14 


10 


0.32 


0.34 


9 ±3.0 


±20 



The noise peak tolerance in the simple binary code transmission of Fig. 
14 was one-half of the marking amplitude. Thus (1 — r)/2 measures the 
ratio of amplitude allocated to the echo to marking amplitude. This is 
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Fig. 17. Phase shift characteristics: (a) remote echo, (6) close-in echo. 



listed in the third column as an echo tolerance. It is converted into 
decibels in the fourth column. 

The tolerance may be placed instead on the ripple amplitude in the am- 
plitude response characteristic. The numerical figure of the third column 
expresses this ripple excursion, in each direction, in nepers. It is con- 
verted, in the fifth column, into decibels. 

The tolerance may also be placed on the phase shift ripple. For this 
purpose the quantity of the third column is assumed as measured in 
radians. For convenience it is converted into degrees in the sixth column. 

So far, nothing has been said concerning the absolute propagation time 
of the system. When this is taken into consideration, it is found that the 
phase ripple really occurs about a diagonal straight line through the origin, 
as illustrated in Fig. 17a. 

Where the echo delay is very short, and only a small portion of a ripple 
cycle appears in the utilized frequency band, the straight line about which 
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the phase deviations are to be taken is not so easily identified. A more 
or less arbitrary, but practical construction is given in Fig. 176. Here 
the straight line is drawn to intersect the actual phase at the frequency 
which marks the top of the nominal effective band. This is the reciprocal 
of twice the signal element duration. The phase departure is taken as 
the maximum double excursion from this straight line, as indicated by 
A4> in the figure. 

Note that in Fig. 17a the excursion A$ measures double the ripple am- 
plitude and consequently double the echo amplitude. In Fig. 176 the double 
ripple amplitude is not accessible within the scope of the plot, and is larger 
than A0. Figure 17a represents a remote echo, such as at B in Fig. 16, 
and Fig. 17b represents a close-up echo, such as at C in Fig. 16. 

It is then clear that for a given excursion A0 the echo amplitude in Fig. 
17b (close-up echo) is larger than for Fig. 17a (remote echo) . 

This variation of the actual echo amplitude for a given A0 corresponds 
approximately to the variation in permissible echo amplitude for a given 
impairment suggested in Fig. 16. Thus (Ref. 7) the specification of the 
phase departure, for an allotted impairment, is roughly independent of 
whether the departure occurs as a single long bend such as in Fig. 176, 
or as a fine structure ripple such as in Fig. 17a. 

The tolerances which are listed in the last two columns of Table 1 were 
set for a binary digital transmission. Tolerances for a continuous analog 
transmission have in practice been found to be of the same order of magni- 
tude. 

However, tolerances for a discrete analog system, with time division 
multiplexed channels interlaced, need to be much more severe (Ref. 27). 

Envelope Delay Distortion. In practice it is often convenient to 
measure the phase shift characteristic of a transmission system in terms 
of its envelope delay. This represents the time of transmission of the 
envelope of a carrier, as the carrier frequency is varied through the spec- 
trum. It is measured as (Ref. 28) 

(12) D = d4>/du, 

where D = envelope delay, seconds, 
<t> = phase shift, radians, 
co = radian frequency, radians per second. 

When this differentiation is applied to the simplified eq. (10) the result is 

(13) D = (d/dco) a sin cor = ar cos cor. 
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The double excursions of the ripples in eq. (13) are 
(14) AD = 2ar. 

It is found from eq. (13) that the application of a fixed tolerance on the 
envelope delay ripple irrespective of the wavelength of the ripple (or cor- 
responding echo delay r) leads to an exaggeratedly severe limitation on 
echo amplitude a for large values of r. 

In other words the use of the envelope delay for the purpose of specifying 
limits on phase distortion for data transmission tends to be unduly severe on 
fine structure excursions in the characteristic. Thus when the envelope delay 
criterion is used, it is necessary to be aware of this and appropriately ignore 
the finer structure ripples in the characteristic. 

In a general way it is found (Ref. 7) that a delay distortion of ±0.4 
signal element duration gives a noise impairment, under unfavorable 
conditions, of about 3 db. If one takes distortions as roughly proportional 
to the permissible echoes the tolerance figures are given in Table 2. 

Table 2. Envelope Delay Tolerances 

Impairment, db Tolerance, Signal Element 
1 ±0.15 

3 ±0.4 

6 ±0.7 

10 ±0.9 

Quadrature Component in Vestigial Sideband Transmission. An 

interfering component similar in certain respects to an echo is generated 
by the usual form of vestigial sideband transmission (Sect. 2, Amplitude 
Modulation). This is the quadrature component, so called because this 
interference adds in quadrature to the otherwise undistorted signal (Ref. 
29). 

Although the precise wave shape of this interfering component is dif- 
ferent from that of an echo, it does use up signal amplitude range in much 
the same manner and requires an increase in signal to noise margin. 

It has been shown (Refs. 29, 30) that the amplitude of this interfering 
component varies according to how much frequency space is allowed to 
the vestigial sideband, and, to some degree, to the particular shape of the 
cutoff. The wider this frequency space, the smaller will be the amplitude 
of the quadrature component. In actual data transmission practice the 
vestigial bandwidth used, as measured from the carrier to the frequency 
at which the response drops to a very low value, tends to run from some 
one-half to one-fourth of the nominal effective band. 
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The amplitude of the quadrature component can also be changed by 
changing the depth of modulation of the signal. The depth of modulation 
is reduced by sending a finite amplitude (instead of the more usual zero 
amplitude) of carrier during a spacing signal. This reduces the quadrature 
component, and to that extent it reduces the impairment which it causes 
in the signal to noise ratio. It does, of course, also reduce the amplitude 
range of the signal between marking and spacing, and to that extent also 
impairs the signal-to-noise ratio. As the spacing carrier rises, this impair- 




0.25 0.50 0.75 

Ratio of spacing to marking signal 



1.0 



Fig. 18. Noise impairment caused by quadrature component in vestigial sideband 

transmission. 



ment also rises, and the quadrature component impairment drops. At 
some value there is a minimum total impairment. A typical case has been 
worked out by Sunde (Ref . 30) and is illustrated in Fig. 18. 

Level Changes 

Transmission systems in general show some variation with time in over- 
all net loss (or gain) . This comes from a variety of causes, such as changes 
in temperature (and therefore resistance) of conductors, variation in 
battery supply, and aging or replacement of vacuum tubes. 

Analog System. An amplitude modulated analog system is especially 
vulnerable to received level change. A system engineered to a possible 
±5 per cent error does not represent a very high performance. Yet if 
all the error is assigned to level change, this is required to be within less 
than ±3^ db. This is a severe requirement for anything but a compara- 
tively short direct wire circuit. 
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Because of this, an analog system usually uses a pilot channel of some 
type to transmit a reference amplitude. Also, frequency modulation is 
often preferred to amplitude modulation. Even in this case, however, in 
some carrier facilities, as was mentioned above in Sect. 2, there is a change 
in frequency which is analogous to a level change. Thus a pilot channel 
may be needed to send a reference frequency. 

Digital System. In a binary digital system a level change cuts into 
the signal-to-noise margin somewhat in the same way as does an echo. 
If no change is assumed in the critical level distinguishing a mark from a 
space, then for the four grades of impairment considered before, the 
allowances are given in Table 3. 



Table 3. Level C 


Change Tolerances 


Impair- 
ment, db 


r = Ampli- 
tude Ratio 


(r + l)/2 = 
Amplitude 
Tolerance 


Tolerance, 
db 


1 

3 

6 

10 


0.89 
0.71 
0.50 
0.32 


0.95 
0.86 
0.75 
0.66 


0.5 
1.4 
2.5 
3.6 



These are still fairly severe requirements, and usually some compensating 
device is provided in the system. This can be an automatic adjustment 
of the critical level at a given fraction of marking level, or an automatic 
volume adjuster for the marking level, or possibly even both. 

If a three-level signal is used, as in Fig. 12, the tolerances are correspond- 
ingly more severe. 
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1. SYMBOLS FOR FEEDBACK CONTROL 

Alphabetical List by Letter Symbols 

Terminology given in Table 1 is for feedback control covered in the 
following chapters. In the case of specific physical examples, the termi- 
nology of the particular field from which the example is taken will be used; 
for example, in an electrical example, e may be used for voltage and i 
for current. The last column of the table lists the chapter where the sym- 
bol is first used. This reference may be useful to the reader for looking up 
discussions of the various quantities. 

The nomenclature used is patterned after the standard nomenclature 
and symbols of the American Standards Association (Ref. 1). Capital 
letters will be used to represent the Laplace transforms of the time func- 
tions; for example, A(s) is the Laplace transform of a(t). An asterisk (*) 
indicates that the quantity is in sampled form; for example, e*(t) is the 
sampled form of the signal e(t). 
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Table 1. Letter Symbols for Feedback Control 

First 
Symbols Use or Term Used 

a Arbitrary constant and/or coefficient for differential 

equation 
A Arbitrary constant for time response equation 

a(t) Impulse response of reference input terms (function Chap. 20 

of time) 
6, B Arbitrary constant for time response equation 

B Magnitude of deadband Chap. 25 

b(t) Primary feedback variable (function of time) Chap. 20 

c(t) Controlled variable (function of time) Chap. 20 

c*(t) Sampled form of c{t) Chap. 26 

d, D Arbitrary constants for time response equation 

D Magnitude of negative deficiency; a denominator 

term; used also as a subscript Chap. 25 

D(s) Polynomial in s usually the denominator Chap. 22 

e(t) Actuating signal (function of time) Chap. 20 

e*(t) Sampled form of e(t) Chap. 26 

/ Frequency, cycles per second (see definition of co) 

f(t) Arbitrary variable (function of time) 

g{t) Impulse response of forward element (function of Chap. 20 

time) 
Gd>( I x | , co) Describing function Chap. 25 

or Gd 
h(t) Impulse response of feedback elements (function of Chap. 20 

time) 
H Magnitude of hysteresis Chap. 25 

i ith term in a series, used as subscript 

i(t) Ideal value of the ultimately controlled variable Chap. 20 

(function of time) 
j Complex number, V— 1 

k kth term in a series, used as a subscript 

K Gain constant for system 

Ko, K\, K.2, Dynamic error coefficients, subscript indicates asso- Chap. 20 

etc. ciated derivative 

K p Static position error coefficient Chap. 20 

K v Static velocity error coefficient Chap. 20 

K a Static acceleration error coefficient Chap. 20 

£ Denotes application of the Laplace transform integral 

£ -1 Inverse Laplace transform 

m Used as a subscript to denote wth term in series 

m(t) Manipulated variable (function of time) Chap. 20 

Chap. 21 



C 
M Magnitude of — (jw), i.e., 

K 



C 'c \ 



M m Maximum value of 



f(ico) 



Chap. 21 



n Use as a subscript to denote nth term 

n(t) Output of nonlinear element 

N Particular value of n; a numerator term; a subscript 

N(s) Polynomial in s usually the numerator Chap. 22 
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Table 1. Letter Symbols for Feedback Control (Continued) 

First 

Symbols Use or Term Used 

p, p 2 Differential operator, p = d/dt, p 2 = d 2 /dt 2 Chap. 20 

p n nth. pole Chap. 22 

p(x) Probability distribution of x Chap. 24 

P Number of poles in right half of s-plane Chap. 21 

P(x = n) Probability function Chap. 24 

q(t) Indirectly controlled variable (function of time) Chap. 20 

r(t) Reference input variable (function of time) Chap. 20 

R Number of counterclockwise rotations of a vector Chap. 21 
from — 1 + jO to H(ja)g(j(S) locus as co varies from 
to j<x> to — joo to — 

S Magnitude or level of saturation Chap. 25 

s Laplace transform operator = <j + joo 

s a , Sb, s c Roots of numerator of G(s), zeros; z n also used in this Chap. 22 

case 

si, S2, S3 Roots of denominator of G(s), poles; p n also used in Chap. 22 

this case 

t Time, seconds 

t r Rise time, seconds Chap. 22 

ta Delay time, seconds Chap. 22 

t p Time to first peak or overshoot of transient, seconds Chap. 22 

t s Settling time, seconds Chap. 22 

T Time constant, seconds Chap. 20 

u(t) Disturbance variable (function of time) ; step function Chap. 20 

v(t) Desired value or command variable (function of time) Chap. 20 

w{t) Impulse response of given element 

x(t) I Variables used when standard 

y(t)\ terminology for feedback systems is not appli- 

cable 

y e (t) System error (function of time) Chap. 20 

yd{t) System deviation (function of time) Chap. 20 

z(t) Indirectly controlled system impulse response Chap. 20 

z z transform operator Chap. 26 

z n nth zero Chap. 22 

Z Number of zeros in right half of s-plane Chap. 21 

a Phase angle of closed loop frequency response Chap. 21 

7 Phase margin Chap. 21 

5 Increment; Dirac function, impulse function Chap. 20 
A Incremental change in variable, usually used Ax, Ay, 

etc. 
A Denotes equality by definition 

£ Relative damping, damping tactor 

6 Phase of angle of open loop frequency response Chap. 21 
X 1 Frequency, damping, used when a and co are not ap- 

£ J plicable 

Tr 3.14159 

II(a;) Product sign meaning X\'Xi- xz -x\ x n Chap. 20 

n 

a Standard deviation (probability) ; Chap. 24 

decrement factor Chap. 20 
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Table 1. Letter Symbols for Feedback Control (Continued) 



First 



Symbols 


Use or Term 


Used 


S(s) 


Summation : xi + £2 + £3 -\ [- x n 




r 


Normalized time 




0(r) 


Autocorrelation function 


Chap. 24 


$(«) 


Power spectral density 


Chap. 24 


* 


Arbitrary angle 




CO 


Angular frequency or natural angular frequency, ra- 
dians/second 




C0& 


Bandpass frequency 




We 


Crossover angular frequency, frequency at which 
\G(s)H(s)\ =1 


Chap. 22 


co m 


Angular frequency at which M m occurs 




co n 


nth frequency term 




COo 


Undamped natural angular frequency 





2. GENERAL FEEDBACK CONTROL SYSTEM DEFINITIONS 

Basic Feedback Control System Elements 

The basic elements of a typical feedback control system and the symbols 
and nomenclature used to describe them are given in Fig. 1 and defined in 
Table 2. The majority of the feedback control systems can be charac- 
terized by these elements although not all the basic elements will exist in 
every system. More complex systems can normally be represented by 
simply expanding the basic elements given in Fig. 1. Basic to any feed- 
back control system are the forward elements, (g), and feedback elements, 
(h). Both the feedback and forward elements as well as the other basic 
elements may consist of subsystems that are feedback control systems 
themselves. 

In cases where the standard terminology does not fit the control system 
under consideration, the individual authors have used the terminology 
they consider best suited. In such cases, the nonstandard terms and 
notation are defined where they are used. 

The symbols and nomenclature of Fig. 1 are based upon Ref. 1. The 
(t) as in a(t) can be dropped when convenient if the simpler symbol will 
not be misinterpreted. The usage of the symbols appearing in Fig. 1 and 
in Table 2 is explained in Table 1. 

The techniques for manipulating block diagrams are given in Chap. 20, 
Sect. 3. 

Alphabetical Index for Definitions of Terms in Feedback Control 

Table 2 is patterned after a similar table in Letter Symbols for Feedback 
Control Systems (Ref. 1). See also Ref. 2. 
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Fig. 1. Block diagram of feedback control system containing all basic elements. 
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Table 2. Definitions 

Term Definition 

Actuating sig- Actuating signal ratio is the transform ratio of the actuating 

nal ratio signal to the reference input. 

Under linear conditions this ratio is expressed as a Laplace 
transform „ .. 

— n = 

R w 1 + <?(«)#(«)' 

Control accu- Control accuracy is the degree of correspondence between the 

racy ultimately controlled variable and the ideal value. 

Control area A control area of a feedback control system is the time integral 

of the absolute value of the difference between the con- 
trolled variable and its final value following a specified step 
input or disturbance. The step input or disturbance must 
be specified in location and magnitude. 

Control pre- Control precision is the degree of reproducibility of the ulti- 

cision mately controlled variable for several independent applica- 

tions of the same reference input under the same operation 
conditions. 

Control ratio Control ratio is the transform ratio of the controlled variable 

to the reference input. Under linear conditions, this ratio 
is expressed mathematically as the Laplace transform 

C 1 

/T S; l + G(s)H(s)' 
Corner fre- The corner frequency of a factor of a transfer function is the 

quency frequency at which lines asymptotic to its log magnitude 

curve intersect. 
Dead time Dead time is a fixed interval of time between the impression 

of an input on an element or system and the undistorted 
response to the input. Often, this is taken as the time to 
reach either 10% or 50% of the total change. 
Error ratio The error ratio is the transform ratio of the system error with 

respect to the reference input. Under linear conditions, 

the ratio is expressed mathematically as -^ (s). In simple 

K 

systems where the system error is equal to the actuating 

signal, the actuating ratio becomes the error ratio, i.e., 

-R (s) = R (s) = T+Wy 

Feedback con- A feedback control system is a control system which tends to 

trol system maintain a prescribed relationship of one system variable 

to another by comparing functions of these variables and 
using the relationship as a means of control. 

Feedback con- The feedback controller is a mechanism which measures the 

troller value of the controlled variable, accepts the value of the 

command, and, as the result of a comparison, manipulates 
the controlled system in order to maintain an established 
relationship between the controlled variable and the com- 
mand. 
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Term 
Frequency- 
response 



Gain, magnitude, 
attenuation 



Gain crossover 
Gain margin 
Input resolution 



Linear system 



Log magnitude- 
angle diagram 

Log magnitude 
and phase dia- 
gram (Bode 
diagram) 

Loop gain 

Loop input (or 
output) reso- 
lution 



Table 2. Definitions (Continued) 

Definition 

The frequency response of a system or element is the steady- 
state ratio of magnitude and the difference in phase of the 
output with respect to a sinusoidal input. The range of 
frequency and conditions of operation and measurement 
must be specified. 

Gain of a system or element is the ratio of magnitude of the 
output with respect to the magnitude of sinusoidal input. 
The frequency and conditions of operation and measurement 
must be specified. 

Gain crossover is a point in the plot of loop frequency response 
at which the magnitude of the loop frequency response is 
unity. 

Gain margin is the amount by which the magnitude of the 
loop ratio of a stable system is different from unity at phase 
crossover. It is frequency expressed in decibels. 

Input resolution between two variables of a system or element 
is the maximum change in the variable considered as the 
input which can be made without causing a change in the 
variable considered as the output. 

Resolution may be dependent upon conditions of operation 
and the operating point. If these are not specified, the 
maximum value of the resolution over the entire operating 
range and for all conditions of operation is implied. 

A system represented by a linear differential equation is a 
linear system. A system may be linear only within a certain 
region of operation. In such cases the boundaries of the 
region should be specified. The term is often used in a 
more limited sense to include only systems described by 
constant coefficient linear differential equations. 

A log magnitude-angle diagram is a plot of the log magnitude 
versus angle of a transfer function with frequency as a 
parameter. 

A log magnitude and phase diagram is a plot of the log magni- 
tude and phase angle of a transfer function versus log fre- 
quency. Log magnitude and frequency may be multiplied 
by constants. 

Loop gain is the magnitude of the loop ratio. 

The loop input (or output) resolution at the specific variable 
is the input (or output) resolution when the loop is opened 
at the specified variable. 

When the loop is opened in a feedback control system, the 
dependent variable shall be called the output and the in- 
dependent variable shall be called the input. When using 
letter symbols, the subscripts o and i shall be used with the 
variable to designate output and input respectively, viz., 
e and ej indicate that the loop is opened at the actuating 
signal. 
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Term 
Loop ratio or 
open loop 
frequency 
response 



Mixing point 



Nonlinear system 

Normal time 
response 

Nyquist dia- 
gram 



Output resolu- 
tion 



Parametric vari- 
ation 

Phase crossover 

Phase margin 

Primary feed- 
back ratio 



Response time, 

transient 

response 
Rise time 



Table 2. Definitions {Continued) 

Definition 
Loop ratio is the transform ratio or frequency response of the 
primary feedback to the actuating signal. Under linear con- 
ditions the ratio is expressed mathematically as the Laplace 
-transform : R 

^(s) = G(s)H(s). 

The mixing point is a symbol used in block diagrams to denote 
the combining of two or more signals so that the output is 
a function of the input signals. The direction of signal flow 
is indicated by arrows. The summing point is a special case 
of the mixing point. 

A system represented by a nonlinear differential equation is 
a nonlinear system. 

The normal time response is the time response with zero initial 
energy storage. 

A Nyquist diagram is a closed polar plot of a loop transfer 
function from which stability may be determined. For a 
single loop system it is a map on the H(s)G(s) -plane of an 
s-plane contour which encloses the entire right half of the 
s-plane. 

Output resolution between two variables of a system or ele- 
ment is the minimum change in the variable considered as 
the output which can be made by changing the variable 
considered as the input. Resolution may be dependent upon 
conditions of operation and the operating point. If these 
are not specified, the maximum value of the resolution over 
the entire operating range and for all conditions of operation 
is implied. 

A parametric variation is a change in system properties which 
affect the performance or operation of the feedback control 
system. 

Phase crossover is a point on the plot of loop ratio at which 
its phase angle is —180°. 

Phase margin is the angle by which the phase of the loop ratio 
of a stable system differs from —180° at gain crossover. 

Primary feedback ratio is the frequency response of the pri- 
mary feedback to the reference input. Under linear condi- 
tions this ratio is expressed mathematically: 

B G(s)H(s) 

R W 1 + G(s)H(s) 

The response time of a system or element is the time required 
for the output to first reach a specified value after the appli- 
cation of a step-input or disturbance. 

The rise time of a system or element is the time required for 
the output to increase from one specified percentage of the 
final value to another following the application of a step 
input. Usually the specified percentages are 10% and 90%. 
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Table 2. Definitions (Continued) 

Term Definition 

Rise time The slope at 50% of the final value is sometimes defined as 

(continued) rise time. 

Root locus The root locus diagram is a graphical method for finding the 

diagram closed loop roots of the characteristic equation. The analy- 

sis starts from the factored roots of the open loop equation 
and uses the complex plane for the manipulation. The root 
locus is the locus of the closed loop roots as a function of 
open loop gain. 
Servomechanism A servomechanism is a feedback control system in which the 

controlled variable is mechanical position. 
Settling time The settling time of a system or element is the time required 

for the absolute value of the difference between the output 
and its final value to become and remain less than a specified 
amount, following the application of a step input or dis- 
turbance. The specified amount is often expressed in terms 
of per cent of the final value (typical values: 2 or 5%). 
Spurious com- A spurious command is an undesired component of the com- 

mand mand. 

Spurious refer- A spurious reference input is an undesired component of the 

ence input reference input. 

Stability Stability is the property of a system or element whose response 

to a stimulus dies down, if the stimulus is removed. 

A statement that a system is stable means that the system 
is stable under all normal operating conditions and for all 
types of stimuli normally encountered. A system may be 
referred to as being stable in one region of operation and not 
in another. If such is the case, the region of stability should 
be specified. 

A system is normally regarded as unstable if its output is 

oscillating in the absence of stimuli, but may be specified 

as being stable if the oscillations are less than a prescribed 

magnitude. 

Steady-state Steady-state error is the error which remains after the transient 

error has expired. 

Stimulus A stimulus is any type of signal which affects the controlled 

variable; for example, reference input and disturbance. 
Summing point The summing point is a descriptive symbol used in block di- 

agrams to denote the algebraic summation of two or more 
signals. The direction of information flow is indicated by- 
arrows and the algebraic nature of the summation by plus 
and minus signs. 
Time constant The time constant of an exponential component of a transient 

response is the time required for the component to decay 
from one value of l/e to that value. 
(a) In an element or system whose response to a step input 
is a first order exponential, the time constant is the output 
change to be completed, divided by the rate of change of 
the output. 
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Term 
Time constant 
(continued) 



Time response 



Total overshoot 

Transfer func- 
tion 



Transfer locus 
Transient error 



Transient over- 
shoot 

Transient 
response 

Ultimately con- 
trolled vari- 
able 



Table 2. Definitions (Continued) 

Definition 
If the response to a step input is not a first order expo- 
nential, this ratio is not constant with time. In such a 
case, the definition in the first paragraph still applies to 
each exponential component of the response. 
(b) Mathematically the time constant is equal to T\ in the 
transform factor (1 + T\s)~ l and in the exponential 
exp (- t/Ti). 
The time response of a system or element is the output as a 
function of time following the application of a prescribed 
input under specified operating conditions. 
The total overshoot is the maximum negative value of the sys- 
tem error for a specified positive stimulus. 
The transfer function of a system or element is the relation- 
ship between the output and the input under specified con- 
ditions. 
In a linear system the transfer function is the ratio of the trans- 
form of the output to the transform of its input under the 
conditions of zero initial energy storage. It is a complete de- 
scription of the dynamic properties of a system and may be 
represented as a mathematical expression, the frequency re- 
sponse, or the time response, to a specified input. 
The transfer locus of a system or element is a complex plot of 

its frequency response. 
The transient error is the difference between the system error 
at any time and the steady-state system error for a specified 
positive stimulus. 
The transient overshoot is the maximum negative value of the 

transient error. 
See time response. 

The ultimately controlled variable is a general term which 
refers to the indirectly controlled variable, or in the absence 
of such a quantity corresponds to the controlled variable. 
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Alternate Symbols Commonly Used 

Table 3 gives a comparison of the preferred nomenclature and some of 
the alternate symbols and nomenclature that may be encountered. See 
Ref. 3. 

Table 3. A Glossary of Alternate Terms and Symbols ° 



Preferred Term 
and Symbol 

Command, v 

Reference input 

elements, a 
Reference input, r 

Actuating signal, e 

Control elements, 

Manipulated vari- 
able, m 

Disturbance, u 

Controlled system, g^ 

Controlled variable, 
c 

Indirectly controlled 
system, z 

Indirectly controlled 
variable, q 

Feedback elements, h 

Primary feedback, b 

Summing point 

Idealized system, gi 

Ideal value, i 
System error, y e 
System deviation, yd 



Other Terms and Symbols Used 

Input, desired value, set point, control point: 0,-, d, 0, i 

as subscript 
Conversion elements, primary element, sensing elements 

Input, reference standard, desired value, set point: 0, v, 
di, d, i 

Error, unbalance, actuating error correction, deviation: 
e, 0, 6 e , u 

Amplifier, controller, servo amplifier, relay, error cor- 
rector: y, kg, h, /x, a 



Load disturbance, upset, noise, drift: I, d, n 

Process, plant, load: y, kg, h, /x, a 

Output, regulated variable, measured variable: 8 , 9, r, 

o as subscript 
Process, plant, load 

Output, regulated variable: d , 6, r, o as subscript 

Feedback transfer function : y, kg, g, /3 

Monitoring feedback, feedback : 6/ 

Error detector, error measuring means, discriminator 

Ideal system, desired system, preferred system, reference 

system 
Desired value 
Error, deviation 
Error, deviation 



« From A.I.E.E. Subcommittee Report (Ref. 3, p. 908). 
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3. FEEDBACK CONTROL SYSTEM DESIGN CONSIDERATIONS 

Usefulness of Feedback 

A feedback control is a closed loop control in contrast to an open loop 
control in which no information is fed back from the output. The closed 
loop type of control potentially has the advantages of providing: 

1. Less dependence upon system component characteristics. 

2. Less sensitivity to load disturbances. 

3. Faster response to command signals. 

4. Extreme modification of system behavior. 

Feedback controls are thus more accurate and the performance is more 
predictable than open loop controls. Feedback controls have found wide 
application where these characteristics are desired. Open loop controls, 
however, are simpler and usually less expensive and are the choice in 
applications where reduced accuracy and performance are not deleterious 
and also where a feedback signal is not available; e.g., in a ballistic missile. 

Major Steps in Feedback Control Design 

Design itself is a feedback process. The engineer will use an iterative 
process in which the following steps are repeated several times. Each 
time the effect of design modifications on the inputs to the design are 
observed, and the resulting refinement converges on the final design. 

1. Determine Process Characteristics. The characteristics of the 
process to be controlled must be adequately understood and described 
either by literal expressions or graphical means. Both the dynamic and 
static characteristics must be represented. It may require extensive 
analytical investigation and/or experimental tests to define properly the 
process characteristics. 

2. Specify System Performance. The desired system performance 
must be specified or defined before synthesis can proceed. The perform- 
ance must be interpreted in terms that can be used in synthesizing the 
controller. Often the customer's requirements will not be in a directly 
usable form and must be translated into a more convenient one. 

Frequently used specifications are indicated in Table 4. The transient 
specifications must be prescribed for a particular input signal. Because 
of the practical difficulties, the signal representation is usually limited to 
simple aperiodic or combinations of aperiodic signals for analytical in- 
vestigation. In cases where this is not sufficient, analog or digital com- 
puters are normally used to determine performance. 

The frequency response and transient specifications are related uniquely 
in a linear system; however, the relations are difficult to manipulate and 
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as a result the specifications are often redundant. No explicit relations 
exist in nonlinear systems, and it is necessary to specify in detail all sig- 
nificant characteristics. 

3. Selection of Power Elements. The task to be performed by an 
automatic control will require certain output power, velocities, and ac- 
celerations, and an average power that is dependent upon the duty cycle. 
This information is quite often not known in detail, and values are assumed. 
These values will determine the selection of the power element. The choice 
of electrical, hydraulic, pneumatic elements must be made on the basis 
of weight, cost, performance, and the ability to be integrated with the 
overall system. Within the limits of the state of the art, the power ele- 
ment will set the major lags of the controller and, for amplifying control 
systems such as a power positioning system, it will constitute the major 
time lags of the system. For this reason the final selection is often deferred 
until further study of the control performance to determine if a lower 
performance and less expensive (in cost and/or weight, size, etc.) unit 
can be used. The environmental conditions to which the equipment will 
be subjected must be considered as well as the performance requirements. 
Under extreme ambients or corrosive atmospheres performance may have 
to be sacrificed in order to obtain the necessary operating life. 

4. Synthesize Controller. Within the context of the defined process 
and the preliminary selection of the power elements the actual control 
design is made. This design must be based upon practical compensating 
networks and feedbacks and the availability of the necessary components 
to meet the system performance requirements. 

The optimum design of control systems is primarily a matter of engineer- 
ing judgment. Although various criteria have been advanced and received 
some application, the complex and diverse factors which influence the 
final selection are not amenable to simple mathematical representation. 
No adequate criterion for use as an index is available. 

In selection of the system compensation, consideration must be given to : 

a. Extraneous Signals. Satisfying the system requirements is straight- 
forward if the input contains only correct information. Unfortunately, 
most input signals will contain noise or other extraneous signals. The 
feedback control must be designed to be selective to the correct signal, 
and in so far as possible it must reject the extraneous signals. The ex- 
traneous signals may occur at points in the system other than the input 
such as load disturbances. Inclusion of these effects may radically in- 
fluence the system design. 

b. Nonlinearities. No system is free from nonlinearities, and these 
effects must be considered before the design is complete. The effect of 
nonlinearities may be to reduce performance or even to cause instability. 



Table 4. Common Performance Specifications 



Type 
1. Transient overshoot 



Specified 
Transient response 



2. Settling time 



Transient response 



3. Steady-state error 



4. Rise time 



Transient response 



Transient response 



5. Dead time 



Transient response 



Definition 
Usually taken as ratio of 
peak of transient to final 
value for a step command 
or disturbance. 



Defined as time to reach 
and remain within a spec- 
ified percentage of final 
value (often as 5% or 
2%) after a step com- 
mand or disturbance. 

Final error existing between 
desired and actual out- 
put. 



Defined as (a) time to \ the 
final value, or (b) slope at 
\ the final value, or (c) 
time between 10% and 
90% of final value after a 
step command. 

Defined as (a) time for out- 
put action to be initiated, 
or (6) for output to reach 
a given level (10% or 
50%), or (c) time to the 
intersection of the slope 



General Remarks 

Convenient when transient solution is avail- 
able. Can be estimated from root locus or 
frequency response. Useful for nonlinear 
systems. System must be underdamped. 
Used for regulators, meters, position servo- 
mechanisms which are normally excited by 
step inputs and are underdamped. 

See 1 above. Used for systems which require 
rapid synchronization, e.g., fire control sys- 
tem. 



See 1 above. Easily calculated from static 
characteristics or final value theorem. Use- 
ful when input is simple aperiodic function. 
Can include frequency components which 
arise in nonlinear systems. 

Easily estimated from frequency response or 
root locus and is indicative of band pass of 
system. Used for overdamped systems. Has 
found application in process controls where 
characteristics (1) or (2) may not be easily 
recognized. 

See 1 above. Easily estimated from frequency 
response and is indicative of phase shift near 
gain crossover in systems. Useful when de- 
lay times exist in system. Used for over- 
damped systems. Both rise and delay time 
derive from filter theory. 
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6. Absolute damping, 
decrement factor 



Transient response 
or root locus 



7. Damping ratio 



Transient response 
or root locus 



8. Load sensitivity 



Transient or fre- 
quency response 



of the transient at \ the 
final value and the initial 
value after a step input. 
Defined as the real part of 
the roots of a quadratic 
system and as such deter- 
mines the rate of decay of 
transient. 



Damping ratio is defined as 
f in the quadratic s 2 + 
2a'of s + co 2 and indicates 
the decay per cycle of the 
natural frequency. 



The sensitivity of the out- 
put to disturbance occur- 
ring at the output or other 
extraneous signals within 
the system. Often ex- 
pressed as the ratio of the 
output for a unit disturb- 
ance to the output for a 
unit signal input. Can 
define steady-state and 
transient sensitivity. 



Convenient method of interpreting more com- 
plex systems in terms of quadratic systems. 
Valuable in combination with relative damp- 
ing in work with root locus analysis. Has 
had extensive use in systems demanding pre- 
scribed transient performance, particularly 
when the time decay is important, e.g., in 
autopilots. 

Useful because it is a parameter in nondimen- 
sional plot of quadratic response. Used in 
combination with 6 above in root locus anal- 
ysis. Used when number and size of over- 
shoot are important. In combination with 6 
above defines decay of oscillatory component 
of transient. 

Useful when analog solutions are available. 
Steady-state specifications can be readily de- 
termined from system equations. If disturb- 
ances can be expressed as frequencies then 
frequency response allows simple interpreta- 
tion. Used in fire control, autopilot, voltage 
regulators, etc., where load disturbances (fir- 
ing torques, wind gusts, load changes) are 
important considerations. 
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Table 4. Common Performance Specifications (Continued) 



Type 
9. Minimum error cri- 
terion 



Specified 
Transient or fre- 
quency response 



10. Phase margin 



11. Gain margin 



12. ilf TO peak 



13. Band width 



Frequency response 



Frequency response 



Frequency response 



Frequency response 



Definition 

The response of the system 
is adjusted to minimize a 
function of the total error 
that results from both sig- 
nal and noise or extrane- 
ous signals. The criter- 
ion may take several 
forms, e.g., min. squared 
error, min. absolute error 
times time. 

Defined as 180°+ phase shift 
at unity gain of the open 
loop frequency response. 

Gain margin is ratio of max- 
imum stable gain to ac- 
tual gain, i.e., gain at 
phase crossover. 

Ratio of maximum of closed 
loop frequency response 
to a low frequency value. 

Defined variously (a) usu- 
ally as frequency where 
closed loop response falls 
to vi or 3 db of its low fre- 
quency value, or (b) some- 
times as the frequency at 
the significant peak M m , 



General Remarks 
Used to optimize system response to reject un- 
wanted noise, and pass the true signal. Used 
to specify performance index when just signal 
is considered. Within basic assumptions 
frequency response analysis is very useful. 
Used on systems which operate on random 
or noisy data, e.g., missile radar guidance and 
fire control. Analog computers can be used 
to apply criterion to nonlinear systems. 

Used as a rule of thumb in frequency response 
analysis to indicate stability and perform- 
ance. Easy to use and to obtain directly from 
frequency response diagram. 

Same as 10. Indicates relative sensitivity of 
system to gain variations. Can be calcu- 
lated by Routh's criterion. Not as good a 
criterion for performance as 10. Little used. 

Used with Nyquist and frequency response 
analysis. Rules of thumb relate M m and 
transient overshoot. Easy to calculate from 
frequency response diagram. 

Used with frequency response analysis and is 
related to speed of response of system. Used 
also when definite frequency bandpass is 
needed for fidelity. M m , bandpass, and the 
phase shift at these valves give a good indi- 
cation of the closed loop response and are 
often used when a number of closed loops are 
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14. Static error coefficient Frequency response 



15. Dynamic error coeffi- 
cients (or steady-state 
error coefficients) 



Frequency response 
and root locus 



16. Maximum system 
error 



Transient response 



or (c) the crossover of the 
open loop response. 

Defined as the final error re- 
sulting from a continu- 
ous input of position, or 
velocity, or acceleration, 
etc. The magnitude of 
the input and the maxi- 
mum tolerable error must 
be specified. 

Defined as the steady-state 
error resulting from the 
derivatives of the input 
function. The time func- 
tion and/or its deriva- 
tives must be specified as 
well as the maximum tol- 
erable error. 



Defined as the maximum 
tolerable system error, y e . 
The input function and 
operating conditions must 
be specified. 



operated in tandem as system. 

Used to set low-frequency gain of open loop 
frequency response. Useful where steady in- 
puts are encountered. 



Relates system gain and time constants to er- 
rors arising from higher derivatives of input. 
Used to estimate error resulting from varying 
input to given system and conversely to de- 
termine closed loops pole-zero location to give 
desired error. Accurate where input varies 
at slow rate compared to bandpass. Becomes 
poorer as input varies more rapidly because 
of transient effects. Used in analysis of fire 
controls, machine controls, etc., where input 
varies in an expected manner. 

Distinguished from steady-state error because 
maximum error under dynamic conditions is 
specified. Normally used to define perform- 
ance with a varying input, e.g., automatic 
milling machine control. Not usually used 
with simple aperiodic inputs. Used in con- 
junction with minimum error criterion (9) 
to place absolute bound on error. 
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Table 4. Common Performance Specifications (Continued) 



Type 
17. Resolution 



18. Duty cycle 



Specified 
Low level charac- 
teristics 



Power element 
rating 



19. Maximum operating 
conditions 



Power element 
rating 



Definition 

Defined as the maximum 
tolerable change in the in- 
put without a change oc- 
curring in the output. In- 
put and operating condi- 
tions must be specified. 

Defined variously, depend- 
ing upon application. In- 
tent is to define the aver- 
age power requirement. 



Included to indicate the 
wide variety of maximum 
performance require- 
ments sometimes speci- 
fied, e.g., maximum ve- 
locity, maximum load 
torque. 



General Remarks 
Can appear in various forms, i.e., maximum 
position input change required to obtain out- 
put change, or minimum velocity at which a 
servomechanism will track with tolerable 
velocity error. 

Objective of specification is to allow more effi- 
cient selection and/or design of the power 
element. Specification can take the form of 
an rms power requirement or where an aver- 
age does not adequately describe the situa- 
tion a time distribution and level may be 
given. Used extensively when large power 
drives are involved. 

Many of these limits are implied by other per- 
formance requirements. Often necessary to 
define implicitly load requirements separate- 
ly, e.g., load running torques or power (inde- 
pendent of accelerating torques) . 
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c. Practical Aspects. The ultimate cost and manufacturability must 
be considered during the synthesis. This, of course, implies ascertaining 
the physical realizability of the controller and assuring that practical 
tolerances are maintained. Reliability and ease of servicing must also 
receive proper consideration. Environmental conditions and customer 
requirements on component packaging must also be factored into the 
mechanical design and may affect the performance. 

5. Test and Evaluation of Equipment. In most cases unpredicted 
and secondary effects will require final adjustment to be made after the 
actual equipment is assembled. This is often the more economical way 
to reach a final design when a wide range of adjustment can be included 
in the design or preliminary models can be built and tested relatively fast. 
This would be the case for many types of instrument servos. On the other 
hand, it would be horrendous to attempt this approach with an elaborate, 
expensive missile system which is expended at each test firing. In such 
cases the extensive use of analysis and computer facilities to minimize the 
testing is justified. 

4. SELECTION OF METHOD OF SYNTHESIS FOR FEEDBACK CONTROLS 

The major analytical methods available to aid in the synthesis of feed- 
back control systems are summarized in Table 5. No general rules are 
available for the selection of the proper method, and the designer should 
be familiar with all methods in order to select the one best suited to his 
problem. It is often desirable to carry root locus and frequency response 
diagrams in parallel. The root locus supplies time domain information, 
and the frequency response provides the simplest method of estimating 
the method of compensation. 

System Optimization 

None of these techniques allows a completely systematic design 
approach. The major difficulty is in defining and specifying optimum 
performance and determining what performance index to use for evalua- 
tion. See Chap. 24. Although criteria have been proposed, the mathe- 
matical labor involved in the more sophisticated is prohibitive. Actually 
the accuracy and extent of the available data usually warrant the use of 
only the more simple criteria. These criteria do not encompass the entire 
problem and therefore must be used carefully. The material in the follow- 
ing chapters presents the available useful design criteria. 
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Table 5. Summary of Major Analytical Techniques for Feedback Control 

System Analysis 

De- 
scribed in 
Chap. 20 



Type 
1. Differential 
equations 



2. RouthHur- 
witz criterion 

3. Root locus 



4. Frequency 
response 



Describing 
functions 



Closed loop 

pole-zero 

location 



Usefulness 

Classical solutions of differential equations are gen- 
erally too involved for practical use in synthesis. 
Nondimensional performance charts help on sec- 
ond order systems. Significance of individual 
system component values difficult to ascertain. 

Used to determine the limiting stability conditions. Chap. 21 
Can be extended to include damping factor only 
with difficulty. Limited usefulness. 

The best solution to the problem of directly syn- Chaps, 
thesizing the time response. Particularly use- 21, 23 

ful when the performance specifications are in 
terms of the time response. Construction of the 
diagrams can be time-consuming and the per- 
formance can be sensitive to small relative 
changes of locus in low-frequency region. 

The most used approach presently available. The Chap. 23 
locus can be plotted in the form of a Nyquist, log 
magnitude-angle diagram, or the log magnitude 
and phase diagram. The latter has the advan- 
tages of easy construction by templates and of 
easy introduction of compensating characteris- 
tics. Easy to include experimental data in fre- 
quency response analysis. The difficulty of re- 
lating transient and frequency response is a lim- 
itation. 

An extension of the frequency response techniques Chap. 25 
to nonlinear systems. Good performance crite- 
rion not available. Method can treat higher or- 
der systems. 

Requires determining realizable and practical com- Chaps, 
ponents after the definition of the system re- 22, 23 

sponse. Not in wide use as yet but possesses 
the good feature of working directly from the de- 
sired closed loop response. 



The Use of Computers 

This has supplanted much of the paper design study. This approach 
allows rapid and complete (often visual) evaluation of the expected system 
performance. At the present state of the art, however, it is not possible 
to obtain a complete design from the computer without interpretation at 
various steps by the design engineer. The ultimate use of the computers 
will occur when a complete systematic design can be programmed; but 
this cannot be done until mathematical expressions can be equated to the 
decisions now based upon "engineering judgment." 
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Availability of computers has not eliminated the need for a thorough 
knowledge of the standard feedback control techniques for analysis. Al- 
though, when an analog computer facility is available, the conventional 
analytical techniques are used principally for preliminary, order-of-magni- 
tude estimates and for verifying computer solutions, experience has shown 
that a thorough knowledge of alternate techniques will enhance the use- 
fulness of the computers. 
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1. REPRESENTATION OF PHYSICAL SYSTEMS 

Methods of System Analysis 

In order to study the performance of a physical system, equations must 
be written from the physics of the situation to describe the excursion of 
all variables. To describe the operation of a physical system in mathe- 
matical form, its differential equations may be written which, in general, 
will be nonlinear in character. In many cases it is possible, by restricting 
the region for which results are valid, to write linear differential equations 
with constant coefficients for the system. The solution of the linear 
differential equation then yields the complete steady-state and transient 
response of the system for a given input. The transient response indicates 
the system stability while the steady-state response to a sinusoidal input 
is very useful in system synthesis. 
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Table 1. Typical Component Equations (Ref. 10) 
Parameter Equation Description 



Translation sys- 
tems: 
Mass 




,, d 2 x 

dx 1 f, . 

Tt = MJ {fl ~ h) 



dt 



The net force acting on a body is 
equal to its mass times its accelera- 
tion with respect to an arbitrary 
fixed reference. 



Spring 

Deformation, 




7 KX 'dt Kdt 



The force which must be applied to 
each end of a spring to deflect it 
a distance x is equal to the spring 
constant K times x. 



Dashpot (vis- 
cous damp- 
ing) 



3 ' ~ \dt dt) 



The force which must be applied to 
each end of a dashpot to produce a 
relative motion of its two ends is 
equal to the viscous damping co- 
efficient D times the relative veloc- 
ity. 



Rotational sys- 
tems: 
Inertia 



qi — ?2 = J 



dd 
dt 



= }/(<?! -ffO 



d 2 . The net torque acting on a body is 

dt 2 ' equal to its inertia times its angu- 

lar acceleration with respect to an 
dt arbitrary fixed reference. 



Torsional 
spring 




Rotational 
dashpot 




q = (?(0i - 2 ) 



q = B 



dd 
dt 



The torque which must be applied 
to each end of a torsional spring to 
produce a relative angular defor- 
mation 0i — 2 of its two ends is 
equal to the rotational spring con- 
stant times the angular deforma- 
tion. 

The torque which must be applied to 
a rotational dashpot to cause it to 
rotate with an angular velocity is 
equal to the rotational viscous 
damping coefficient times the an- 
gular velocity. 
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Table 1. 
Parameter 

Electrical sys- 
tems : 
Inductance 



Typical Component Equations (Ref. 10) (Continued) 



Vi — V2 = L 



-ih 



Equation Description 

The voltage drop caused by current 

flowing in an inductance is equal 

to the inductance times the rate 

of change of the net current flow- 

Vi — V2) at m g m t ne direction of the drop. 



di 
di' 



Capacitance Vi — Vz = — I i dt; 

c 



The voltage drop caused by current 
flowing through a capacitance is 
equal to the integral of the net 
current flowing through the capac- 
itance divided by its capacitance. 



Resistance v\ — V2 = Ri; 



The voltage drop caused by current 
flowing through a resistance is 
equal to the net current flowing 
through the resistance multiplied 
by the resistance. 





English Gravitational Units 






Electrical Units 


t 


time, seconds 


Q 


torque, lb-ft 




t 


time, seconds 


X 


distance, feet 


J 


inertia, slug-ft 2 




V 


voltage, volts 


M 


mass, slugs 


e 


angle, radians 




i 


current, amperes 


K 


spring constant, 


G 


torsional spring 


con- 


L 


inductance, henrys 




lb/ft 




stant, lb-ft/rad 




C 


capacitance, farads 


D 


damping coeffi- 


B 


rotational damping co- 


R 


resistance, ohms 




cient, lb/ft/sec 




efficient, lb-ft/ra 


Ld/sec 







The solution to differential equations by either the direct method or 
by Laplace transformations is useful primarily in the analysis of a given 
system with all parameters prescribed. This approach is less useful in 
the design or synthesis of a control since the effect of the variations of 
parameters on the exponential time function exponents is difficult to visu- 
alize. For more complex systems the problem of factoring the high order 
polynomial characteristic equation becomes quite laborious. 

For synthesis the root locus, frequency response, and closed loop pole 
zero location methods are recommended. (Chaps. 21, 22, and 23.) 

Even for analysis the classical time solution has been largely supplanted 
by the wide usage and availability of analog computers. As a result the 
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classical techniques of solution are used primarily as checks on analog 
computer results or as aids in visualizing the basic performance. The 
charts included in Sect. 2 are useful in this case. 

Although the solution of differential equations is no longer of paramount 
importance, the correct description of the system or component dynamic 
performance by differential equations is basic to all methods of analysis and 
synthesis. It is most important that the control designer understand dif- 
ferential equations and their application to his field of endeavor. 

A suggested approach for obtaining these physical equations is: 

1. Understand the system well enough to draw a schematic diagram 
showing the relationship of all variables, including all pertinent components 
as well as the load. 

2. Replace the schematic with equivalent circuits or analogies. 

3. Rearrange this diagram into convenient noninteracting sections or 
blocks. 

4. Write the characteristic equation of each section from the functional 
relationship. 

5. Obtain the transfer function from these equations. 

6. Simplify this block diagram and obtain the complete system charac- 
teristic equation by algebraic manipulation. 

This sequence is an analysis approach; synthesis of a system reverses 
this method after starting with known requirements to obtain a system 
equation. 

Physical Laws. To write the equations which mathematically describe 
the system or component performance, it is necessary to understand the 
basic operation of the device and the physical laws governing the various 
processes involved. The wide field of application of feedback control 
theory makes it prohibitive to list all the fundamental laws that might be 
required. The following partial list of textbooks in the particular field 
of interest for these physical laws and Table 1 are useful. 

1. Physics: Erich Hausmann and E. P. Slack, Physics, Van Nostrand, Princeton, 
N. J., 1948. 

2. Electrical: W. L. Everitt, Communication Engineering, McGraw-Hill, New York, 
1937. 

3. Thermodynamics: P. J. Kiefer and M. C. Stuart, Principles of Engineering Thermo- 
dynamics, Wiley, New York, 1954. 

4. Fluid Mechanics: R. C. Binder, Fluid Mechanics, Prentice-Hall, New York, 1949. 

5. Kinematics: J. L. Synge and B. A. Griffith, Principles of Mechanics, McGraw-Hill, 
New York, 1949. 

6. Circuit Analysis: E. A. Guillemin, Mathematics of Circuit Analysis, Wiley, New 
York, 1949. 

7. Materials: Stephan Timoshenko, Strength of Materials, McGraw-Hill, New York, 
1953. 



FUNDAMENTALS OF SYSTEM ANALYSIS 



20-05 



8. Hydrodynamics: H. Lamb, Hydrodynamics, The University Press, Cambridge, 
England, 1932. 

9. Mechanics: F. B. Seely, Analytical Mechanics for Engineers, Wiley, New York, 1952. 

Examples. The following examples illustrate the use of basic physical 
laws and Table 1 in obtaining the equations describing the system perform- 
ance. Whenever possible, simplifying initial conditions are chosen. 



■O T3- 



R 



6 



E cos oit 



Ht) 




L 



Fig. 1. Electric circuit. 



1. An electric circuit such as Fig. 1 requires: 

Kirchhoff's Law. The summation of voltage drops in a closed loop is 
equal to zero. 



(1) 



di(t) 
E cos at = Ri(t) + L — — + 

w dt C 



i r 
-j m 



dt. 



M 



Spring 

l/l/l/l 



^- Viscous damping, D 
Fig. 2. Damped spring mass system. 



2. A spring mass system such as Fig. 2 requires: 

Newton's Law. The summation of forces acting on a body equals the 
change in momentum. 

d x dot 

(2) M — = -D Kx or (Ms 2 + Ds -f K)x = 0. 

dtr dt 



3. A combined electrical and rotational mechanical system is a d-c motor 
with fixed field excitation (ignoring armature inductance) and a pure 
inertia load is shown in Fig. 3. 
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Fig. 3. D-c motor with inertia load. 

Summing voltage drops : 

(3) E(t) = Ri(t) + K e N(t), 

where K e = motor voltage constant, 
N(t) = motor speed. 
Summing torques: 



(4) 



dN(t) 
J—^ = K t i{t), 
at 



where Kt = motor torque constant, 
J = motor inertia. 

Eliminating i(t) from eqs. (3) and (4) results in the transfer function of 
output speed to input voltage: 



(5) 



1 



N(t) 



E(t) / RJ 

KA — s+1 



1 



\K e K t ) 



K e (T m s + 1) 



where s = d/dt, 

T m = RJ/K e K t = time constant. 
4. A common electromechanical system is a synchro with a pure inertia 



-{t—vA^ 

01 K t 



02 J 



Fig. 4. Schematic of synchro system. 

load and with viscous damping as shown schematically in Fig. 4, where K t 
is the torque gradient. 
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Summing torques: 

(°) J tt + B ~r = K ^ ~ '«) or ( Js * + Bs + K *)°2 = K ^- 

dr dt 

Further examples are used throughout this section. 

Circuit Simplification Techniques 

Analogies are useful in setting up physical systems and interpreting their 
boundary conditions since this approach compares known systems with 
the unknown. Often thermal, mechanical, hydraulic, etc., systems are 
converted to an electrical equivalent since electric circuit analysis methods 
have been developed to a high degree. Examples of conversions of physical 
systems to electrical equivalents are given in Ref. 2. 

Analogies. The following equations show the analogies among three 
systems: 



Equations System 

d 2 x dx 

~d¥ dt 



(7) M — + D — + K a x = f(t) Mechanical translatory system 



d 2 6 dd 

dt 2 dt 



(8) J— + F h K t d = f(t) Mechanical rotation system 



d 2 q do 

-i + R— . 

dt 2 dt C 



(9) L— 7; + R— + -= f(t) Electric circuit 



where M = mass, slugs, 

D = damping, lb/ft/sec, 
K s = spring gradient, lb/ft, 

x = distance, ft, 
K t = torque gradient, lb/ft, 
= angular displacement, rad, 
L = inductance, henrys, 
R = resistance, ohms, 
J = inertia, slug-ft 2 , 
F = friction, lb/rad/sec, 
C = capacitance, farad, 
q = charge, coulombs. 

Analogous elements are listed in Table 2. 



Electrical elements 



Table 2. Analogous Elements (Ref. 2) 



Electrical resistor 






Electrical capacitor 

T c 



Electrical inductor 



10 
p 

o 

00 



Mechanical elements (translational) 



Er = R 



d(qi - q 2 ) 



dt 



R = resistance 
q = charge 



Viscous damper 



-i' 



f=D 



d(x\ — x 2 ) 
dt 



x = displacement 

D = damping coefficient 



Ec = ^ (qi - q*) 



C = capacitance 



Spring 



I 



A J 



.J* 



/ = K(xi — x 2 ) 



K = spring constant 



L = inductance 



Inertia 



i, r d 2 x 



M = inertia 



m 
O 

CO 
> 

n 

o 
O 

z 



o 



Mechanical elements (rotational) 



Torsional damper 



Shaft stiffness 



Inertia 



[—1—1 



tH fo 



=®* 



r = # 



eft 



5 = damping coefficient 



t = K{e 1 - e 2 ) 



K = stiffness coefficient 



T = J 



dt 2 



J = moment of inertia 



> 

m 

Z 
—i 
> 

C/i 

O 



Hydraulic elements 



Fluid resistance 



P = R h q = R h 



dQ 
dt 



Fluid capacity 



<?,— » c> g,, __ 



P = 



-< 
co 

—i 
m 

> 

z 
> 

CO 



Rh = hydraulic resistance 
q = rate of flow 
Q = quantity of flow 

(Q2 = in electrical analog) 



Ch = hydraulic capacity 
Qi = quantity of inflow 
Q2 = quantity of outflow 
P = pressure 



►o 
o 
o 

S3 
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Fig. 5. Wye-delta transformation. 

Aids for Circuit Simplification. The following techniques are useful 
in reducing the system equations to simpler form : 

Wye-Delta Transformation. The circuits of Fig. 5 are equivalent if 
the following relations are satisfied: 



Z h Z 



b^c 



(10) 


Z x = 

Z a + Zb -\~ Z c 


(11) 


v Z a Z c 


Z 2 — 

Z a + Zb + Z c 


(12) 


v ZaZj, 


Z a + Zb + Z c 



(13) 
(14) 
(15) 



Z a = 



Z h = 



Z r = 



Z\Z 2 + Z 2 Z 3 -f- Z3Z1 





Zl 




Z\Z 2 


+ Z 2 Z% 


+ z 3 z x 




z 2 




Z\Z 2 


-\- z 2 z% 


+ Z 3 Zi 



Superposition. 7/ a system is linear the system response to several inputs 
will be the sum of the response to each input separately (refer to Fig. 6). 



/ 



K 







Linear system 
characteristic 
equation = g 


g(x) + g(y) + g(z) + g(w) 
















^ 


Output 











Fig. 6. Superposition. 



Thevenin's Theorem. The effect of any impedance element in a circuit 
may be determined by replacing all the voltage sources by a single equivalent 
voltage source and all other impedances by a signal impedance in series with 
the impedance of interest. For Fig. 7, the equivalent voltage E a b is equal 
to the open circuit voltage that is present across a-b with the circuit broken 
at a-b. 
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%ab 



Network 


a 


b 


containing 








impedances 




Z 




and voltage 








sources 




T 









1 



Fig. 7. Thevenin's theorem. 

Norton's Theorem. The current in any impedance Zr, connected to two 
terminals of a network, is the same as if Zr were connected to a constant-current 
generator whose generated current (Isc) is equal to the current which flows 
through the two terminals when these terminals are short-circuited, the constant- 
current generator being in shunt with an impedance equal to the impedance of 

Z\2 



Network ^-^ 
containing 
impedances 
and voltage 
sources 

- © 






'■) 


1 

T 








z»( v - 



■©-J 



L^-J 



Z12 Z R 

© 

Fig. 8. Equivalent circuits using Norton's theorem. 

the network looking back from the terminals in question. This theorem is 
similar in many respects to Thevenin's theorem. It is illustrated by 
Fig. 8. 

Nodal and Mesh Analysis. A general approach to circuit analysis is 
illustrated by Fig. 9a, 6 and eqs. (16) through (21). In the nodal analysis 







*\ 



12 



© 




Z 2 



12 




(a) (b) 

Fig. 9. (o) Nodal approach; (6) mesh or loop approach. 

the summation of currents at a junction or node is equal to zero. This is 
useful in solving for an unknown voltage, given driving voltages and im- 
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pedances. 


From Fig. 9a, 


(16) 


i\ — *2 — *3 = 0, 


(17) 


e — e\ e\ e\ 
2i\ Z 2 Z% 


^e^ 


eZ 2 Z z 



Z 2 Z% + Z\Z 2 + ZiZ% 

Use of the mesh analysis uses voltage summations about the closed loops. 
Usually the unknown current, such as i 2 of Fig. 9& is found in terms of 
the known voltages and impedances. From Fig. 96, 

(19) e = Z x i x + Z 2 (i 1 - i 2 ) = (Zx + Z 2 )i 1 - Z 2 i 2 , 

(20) = Z 2 (i 2 - i x ) + Z s i 2 = -Z 2 i x -f (Z 2 + Z 3 )i 2 
to solve for i 2 by using determinates : 



(21) 



*2 



Z x + Z 2 e 
-Z 2 



eZ, 



^i ~\r Z 2 —Z 2 
— Z 2 Z 2 + Z% 



^1^2 "f" Z 2 Z% + ^1^3 



Tables of Typical Transfer Functions. The transfer function of a 
system or element is the ratio of the transform of the output to the transform 
of its input under the conditions of zero initial energy storage. It is a com- 
plete description of the dynamic properties of a system and may be repre- 
sented as a mathematical expression of the frequency response, or the time 
response to a specified input. 

In Tables 3 to 5 are summarized typical transfer functions in Laplace 
transform form for typical mechanical, electrical, and hydraulic control 
elements. For a more complete tabulation of transfer functions of RC 
networks see Chap. 23, Sect. 2. 

Tables 6 to 8 consist of three sections of a morphological table of servo 
components appearing in Ref. 11. 

Further material on the subject of transfer functions may be found in 
Refs. 3, 11, 12. 



Table 3. Summary of Transfer Functions for Representative Mechanical Elements (Ref. 3d) 



Mechanical Elements: Transfer Functions 



Rotation 



Spring mass damper 



Translation 
Spring mass damper 

Spring-dashpot (lag) 

Spring-dashpot (lead) 



Qi(s) = 1/n 

G m (s) (j/K s )s 2 + (B/K s )s + 1 



X(s) _ 1 

Y(s) {M/K)s 2 + (D/K)s + 1 

1 



X(s) 
Y(s) 

X(s) 
Y(s) 



(D/K s )s + 1 

(D/K s )s 
(D/K s )s + 1 



Rotation 




Translation 

Mass 



Damper 



Spring 
K 



x(t) 



yU) 



i \ Platform 



Displacement 
reference 



Nomenclature 

di = load angular position, radians, 
6 m = motor position, radians, 
/ = moment of inertia, pound- 
foot-seconds/second, 
n = gear ratio, 
K s = shaft spring constant, pound- 
feet/radian, 
B = damping torque coefficient, 
pound-feet/radian/second. 



x = mass displacement, feet, 
y = platform displacement, feet, 
M = mass, pound-seconds/second/ 

foot, 
D = damping coefficient, pounds/ 
foot/second, 
K, K s = spring constant, pounds/foot. 



c 
Z 

o 
> 



z 
> 

r- 

O 



-< 

CO 



> 



-< 
</> 

to 



►o 
o 



Table 4. Summary of Transfer Functions for Representative Electric Elements (Ref. 3a) 

Nomenclature 



Electric Elements: Transfer Functions 
D-C motor 
For speed control 

N(s) _ 1 

V a (8) ~ K e (T m s + 1) 
For position control 

e(«) i 



V a (s) Kes(T m s + 1) 



D-C Motor 




v a (t) 



N = velocity of motor, radians/second, 
6 = position of motor, radians, 
V a = applied voltage, volts, 

K e = voltage constant of motor, volts/radians/sec- 
ond, 
T m = motor time constant, seconds. 



O 



D-C generator and motor 
For position control 

0(s) K g /K e R 



D-C Generator 



E c (s) s(T f s + l)(T m s + 1) 



Galvanometer 



#i 



/(*) Js*(T lS + 1) 



nWAi 



e c U) 



rCH. 



Galvanometer 




Constant 
flux 



Kg = generator voltage constant, volts/field ampere, 
R = series resistance of motor and generator arma- 
ture circuit, ohms, 
Tf = generator field time constant, seconds, 
E c = voltage applied to generator control field, volts, 
V a = voltage across drive motor terminals, volts. 



J = moment of inertia of galvanometer element, 
pound-foot-seconds/second, 
Ki = torque coefficient, pound-feet/ampere, 
Ti = time constant of galvanometer coil circuit, sec- 
onds, 
6 = position of galvanometer element, radians, 
/ = signal current, amperes. 



m 
O 

> 

n 

n 
O 

z 

SO 
O 



Gyroscope 



Gyroscope 



Q(s) = K 2 
I(s) Js(T 2 s + 1 



Stabilizing networks 
For rate signals (phase lead) 
E (s) = Ts 
E in {s) ~ Ts + 1 

For integral signals (phase lag) 
E (s) = 1 
E in {s) Ts + 1 

For rate and integral (lead-lag) 
E (s) _ TjTts 2 + (7\ + T 2 )s + 1 
E in (s) T^s 2 + (T 7 ! + T 2 + ri 2 )s + l' 
2 7 i2»T 1 + r 2 



7 — > 




Constant 
flux 



Phase Lead 



Phase Lag 



. jVW/ 1 




/ = moment of inertia of gyroscope, pound-foot- 
seconds/second, 
S2 = angular velocity of gyroscope, radians/second, 
JS2 = angular momentum of gyroscope, pound-foot- 
second, 
K 2 = torque coefficient, pound-foot/ampere, 
T 2 = time constant of gyroscope precession coil cir- 
cuit, seconds, 
I = signal current, amperes. 



Ein = input voltage, volts, 
E = output voltage, volts, 
T = RC, time constant, seconds. 



T = RC, time constant, seconds. 



T 1 = R 1 Cij 

j 2 = R 2 C 2 1 

T\ 2 = .R1C2I 



= time constants, seconds. 



c 
z 
a 
> 

m 

Z 



CO 

-< 
en 



> 

Z 

> 

r— 
-< 
CO 

CO 



►o 
o 



K5 
O 



CN 



Table 5. Summary of Transfer Functions of Representative Hydraulic Elements (Ref. 3a) 

Nomenclature 



= C 2 



Hydraulic Elements 
Valve-piston 
Load reaction negligible 

X(s) = Ci 
Y(s) s 
Spring load dominant 
X(s) 
Y(s) 

Valve-piston linkage 
For phase lag 

X(s) _ b/a 
Y(s) ~ T v s + 1 

For phase lead 

X(S) _ d f TV+l l . rp . rp 

W)~TTblfJTii > l3>Iv 



Valve-Piston Linkage 

Inlet 
r Drain I Drain 

lit 




x = piston displacement from neutral, feet, 
y = input displacement from neutral, feet, 
Ci = piston velocity per valve displacement, second -1 , 
C2 = piston travel per valve displacement. 

For phase lag 
a,b = linkage distances, feet, 

T v = — -— , valve effective time constant, seconds. 
aCi 

For phase lead 

l,b,d = linkage distances, feet, 

Tv = t, — , n/< , valve time constant, seconds, 



(1 + b)C 

T 3 = TviX + b ^ b + d) , lead time constant, 
onds. 



sec- 



D 

DO 
> 

n 

n 
O 

z 

TO 

o 



Hydraulic motor 
With compressibility 

6(g) = S p /d, 



Hydraulic Motor 



VJ 



LJ 



BdJ s2 + dJ S + 1 \ 



With negligible compressibility 
6(8) = S p /d m 
Y(s) VLJ 




m^) 



6 = motor position, radians, 

y = displacement of pump stroke from neutral, feet, 

S p = flow, cubic feet/second, from pump per unit dis- 
placement, y, feet, 

d m = motor displacement, cubic feet, 
J = moment of inertia, pound-foot-seconds/second, 
L = leakage coefficient, cubic feet/second/pound/ 

square foot, 
V = total oil volume under compression, cubic feet, 
B = oil bulk modulus, pounds/square foot. 



c 
Z 

o 
> 



> 

r- 

O 



en 

-< 
en 



> 

Z 
> 

r- 
-< 
C£ 



O 
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Table 6. Error Detectors (Ref. 11) 



(a) Type 

(6) Main 

No. Application 

1 (a) D-c or a-c 
resistance 
bridge 
(6) Position con- 
trol 



(a) D-c tachom- 
eter bridge 

(6) Speed con- 
trol 

(a) A-c mag- 
netic bridge 

(6) Position con- 
trol, partic- 
ularly for 
gyro pickups 
where very 
small forces 
prevail 

(a) A-c synchro- 
system 

(b) Position con- 
trol where 
continuous 
rotation is 
desired 



Possible 
Operation Modifications 

Error voltage, x, appears when the position of the Potentiometer can 

moving arms of the potentiometers A and B are be wound on a helix 

not matched. The power source, E, is applied to get more than 

across both potentiometers. A measures reference 360° of rotation, 
position as voltage and B regulated position as 
voltage, their difference being x. 

Error voltage, x, appears when speeds of tachom- A can be replaced by 

eters A and B vary. A measures reference speed a battery as the ref- 

as a voltage and B regulated speed as a voltage, erence. 
The difference between these voltages is x. 

Error voltage, x, appears when relative positions Four poles instead of 
of rotor A and stator B do not match. Rotor A three can be used 
measures reference position magnetically and sta- with two having ex- 
tor B regulated position magnetically. Voltage E, citing windings and 
across exciting coil, L, provides energy. When two pickup coils 
rotor covers unequal areas of each exposed stator connected bucking, 
pole (unbalanced magnetic bridge) pickup coils M 
and N have unequal voltages induced. Voltage 
difference is x. 

Error voltage, x, appears whenever the relative A dual system can be 

positions of the rotors of synchro-generator, A, used whereby the 

and synchro-control transformer, B, are not unity synchrosystem 

matched. The reference position is measured by sets the approximate 

A as a magnetic flux pattern which is transmitted position and the 

to the synchro-control transformer through the high-speed or vernier 

interconnected stator windings. If the rotor of B system sets the ac- 

is not exactly 90° from the transmitted flux pat- curate position, 
tern, x is produced. 



Operating Features 

A and B can be remote. 
Continuous rotation not 



Accuracy 
Limited by 

Potentiometer 
winding. 



A and B can be remote. Tachometer accu- 
Top speed limited by racy. Commu- 

commutator. tator resistance. 



Limited rotation. Air 
gap usually small. 



Machining toler- 
ance, magnetic 
fringing, and volt- 
age phase shift. 



Unlimited rotation. The Machining toler- 

synchro-generator and ance, accuracy of 

control transformer can winding distribu- 

can be remote. tion. 



Features Deter- Frequently Used 
mining Energy with This Device: 

Required to Vary (a) Table 7 Amplifier 

Reference Quan- (6) Table 8 Error 
tity Measurement Corrector 



Contact arm and 
bushing friction. 



Brush and bearing 
friction. 



Load taken from x. 
Bearing friction. 



Distributed or non- 
distributed winding 
of control trans- 
former rotor. Load 
taken from x. 
Bearing and slip 
ring friction. 



(a) 2, 3, 4, 5 
(6) 1, 2, 3 



(a) 2, 3, 4 
(&) 1. 2, 3 



(a) 2, 4 
(6) 1, 2, 3 



(a) 2, 4 
(&) 1, 2, 3 



m 
D 

DO 

> 

n 

n 
O 

z 

—t 

TO 
O 



(a) Frequency 
bridge 

(b) Frequency 
control 



(a) Millivolt 
bridge 

(b) Tempera- 
ture control 



Error voltage, x, appears when reference and regu- 
lated frequencies differ. Tube channel A produces 
a filtered sawtooth wave that gives a d-c voltage 
inversely proportional to the reference frequency. 
Tube channel B produces a similar voltage as a 
measure of the regulated frequency. The differ- 
ence of these d-c voltages is x. 

Error voltage, x, appears whenever the regulated 
temperature differs from the reference temperature. 
The regulated temperature is measured as a volt- 
age by the thermoelectric effect of two dissimilar 
metals, B. The reference temperature is repre- 
sented as a voltage from the battery-potentiom- 
eter source A. Thedifferenceinthese voltages is x. 



May be used as a 
speed regulator if B 
is made an a-c ta- 
chometer. 



An electronic volt- 
age source or an- 
other thermocouple 
can be substituted 
for A. 



A and B can be remote. 
Tubes can be either gas or 
vacuum. A wide range 
of frequencies can be cov- 
ered. Vacuum tubes 
should be used for high 
frequencies. 



Temperature and 
aging effects on 
tube and circuit 
elements. 



Tube input im- 
pedance. 



(a) 4 
(6) 1, 3 



A and B can be remote 


Ability to detect 


Contact arm and 


(a) 2, 4 




A wide range of tempera- 


very low millivolt 


bushing friction. 


(b) 1, 6 


-n 

C 


ture can be covered. 


signals. 


If electronic volt- 








age source A is 




Z 






used, tube input 




a 
> 






impedance. 





E A- 





© 



U~l 




© 




©Synchro Synchro control 
generator transformer 



© 





Thermocouple ( 







> 

Z 
> 

r— 
-< 



to 

O 



(a) Type 

(6) Main 

No. Application 

7 (a) Phototube 

bridge 
(b) Position con- 
trol by inter- 
cepting a 
light beam 

8 (a) Mechanical 

differential 
(6) Position con- 
trol and 
speed con- 
trol 



(a) Beam bal- 
ance 

(6) Voltage con- 
trol, speed 
control, and 
tension con- 
trol 

(a) Modified 
beam bal- 
ance 

(6) Speed con- 
trol (flyball 
governors) 



Table 6. Error Detectors (Continued) 



K3 

o 

K3 
O 



Operation 

Error voltage, z, appears when movable shutter is 
in other than desired position. Light reaching 
phototube, B, measures shutter position. This 
light is measured as a voltage by the phototube 
current variation . A reference position of the shut- 
ter is represented by the battery-potentiometer 
voltage. The difference of these voltages is x. 

Displacement x appears whenever the relative 
reference and regulated positions change. Refer- 
ence position is measured as an angle by one side 
of the differential A and regulated position as an 
angle by the other side of the differential B. The 
difference in the two positions rotates the middle 
member of the differential giving displacement x. 



Displacement x appears whenever the variable 
force is different from the reference force. The 
variable force, B, and the reference spring force, 
A, are measured as moments. The difference in 
these moments produces displacement x. 



Displacement x appears when regulated speed, o>, 
differs from reference speed. This is represented 
by spring force, A, about fulcrum, 0, the regu- 
lated speed by centrifugal force of mass, B, about 
0. Difference in moments of forces about pro- 
duces displacement x. 



Possible 
Modifications 

An electronic volt- 
age source or another 
light source and pho- 
totube can be substi- 
tuted for A. 



Operating Features 

A and B can be remote. 
Glass surfaces through 
which light travels must 
be kept clean. 



Accuracy 
Limited by 

Continued accu- 
racy of light source 
and phototube. 



Features Deter- Frequently Used 

mining Energy with This Device: 

Required to Vary (a) Table 7 Amplifier 

Reference Quan- (6) Table 8 Error 
tity Measurement Corrector 



Contact arm and 
bushing friction. 
If electronic voltage 
source A is used, 
tube input imped- 



(a) 2, 4 
(6)1 



Spur-gear differen- Since A and B must be lo- Gearing backlash. Power taken from (a) 1, 6, 7, 8 



tial. 



Any variable force 
other than a spring 
can be used. 



Any variable force 
other than a spring 
can be used. 



cated together, synchro 
ties or their equivalent 
can be used to transmit 
remote positions to A 
and B. Continuous ro- 
tation possible with speed 
limited by gears. 

For remote operation B 
can be a transmitted 
force, x movement lim- 
ited. By changing 
springs a wide force range 
can be covered. 



A wide speed range can 
be covered, x movement 
limited. 



x. Bearing fric- 
tion. Pitch of 
gears. 



Load taken from 
x. Friction. 



Magnitude of 
forces. Screw 
pitch and friction. 



(6) 1, 2, 3, 4, 5 



O 
DO 
> 

n 

n 
O 

z 

TO 
O 



Load taken from 


Magnitude of 


(a) 1, 6, 7 


x. Bearing fric- 


forces. Screw 


(6) 1, 3, 5 


tion. 


pitch and friction. 





(a) 1, 7, 8 
(6) 1, 5 



(a) Bimetal Displacement x appears whenever the surround- 

(6) Temperature ing temperature and the reference temperature are 
control different. The reference temperature is repre- 

sented by the position of the adjustable reference 
point, A. The surrounding temperature is meas- 
ured by the position of the bimetal strip, B. The 
difference in these positions produces displace- 
ment x. 



Bimetal can be made 
snap acting at some 
standard tempera- 
ture. 



Wide temperature range 
possible by selection of 
proper bimetal. 



Load taken from Mounting of refer- 
x. Ability to ence point, 

measure accurate- 
ly small x deflec- 
tion. Time lag and 
hysteresis of bi- 
metal. 



(a) 1, 6 
(6) 1, 6 



(a) Float Displacement x appears when regulated and ref- A float controlling a 

(fc) Liquid level erence liquid levels differ. Point A is reference, pulley system can be 

control The liquid level is measured as a position by the used rather than a 

float B. The difference produces displacement x. lever. 



With the proper mechan- 
ical arrangement a wide 
variation in liquid height 
can be controlled. 



Phototube, B 






i (1 Moveable shutter 

\t 



R 



u 




Reference___^^ 
point 



© 



Reference 
spring, A 




Bimetal, B 



Adjustable / 
reference' 
point, A- 



Screw ,K Mass, B— 9 
adjustment \^ £?\/ 

— ^ \Reference 
(10) point 



© 



immmmzmzm 



Load taken from 
x. Variable den- 
sity of the liquid. 
Friction. 



Mounting of refer- 
ence point. 



(a) 2, 7 

(b) 3. 4 




Reference spring, 
force, A 

t <* 

r Reference 

point 



Adjustable 
reference 
point, A 




Float, B 
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to 
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to 
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> 
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to 
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Table 6. Error Detectors (Continued) 



(a) Type 

(6) Main 

No. Application 



13 



(a) Bellows 
(6) Pressure 
control and 
temperature 
control 



(a) Piston 
(6) Pressure 
control 



Operation 

Displacement x appears when surrounding and 
reference pressures differ. Reference pressure is 
represented as position by adjustable point, A. 
Surrounding pressure is measured by the bellows 
as a position. Difference in these positions pro- 
duces displacement x. 

Displacement x appears when regulated pressure 
outputs of pump and reference differ. Reference 
pressure is a force on the piston by spring, A . Reg- 
ulated pressure is a force on the piston by the fluid. 
Their difference produces displacement x. 



Possible 
Modifications 

Spring can be added 
in addition to bel- 
lows spring. 



A standard pressure 
source can be substi- 
tuted for the spring. 



Operating Features 
Limited x travel. 



A and B can be remote. 
Limited x travel. 




<^ 



Piston, B 



Adjustable 
reference 
point, A 



X 



Bellows, B 



Surrounding 
pressure 



Accuracy 
Limited by 

Load taken from 
x. Hysteresis of 
bellows spring. 



Friction. Load 
taken from x. 



Features Deter- Frequently Used 

mining Energy with This Device: 

Required to Vary (a) Table 7 Amplifier 

Reference Quan- (6) Table 8 Error 

tity Measurement Corrector 



K3 
O 

K3 
rO 



Mounting of refer- 
ence point. 



(a) 1, 4, 5, 6 
(« 1, 6 



Piston forces in- (a) 7, 8 

volved. Screw (6) 5 

pitch and friction. 



Screw 
adjustment 



Variable 

pressure 

pump 




00 

> 

n 

n 
O 

z 

—i 

TO 
O 



Standard 
reference 
spring, A 



Reference 
point 
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Methods of Evaluating Transfer Functions Experimentally 

In the analysis or design of feedback controls and when equipment is 
available, it may be desirable to measure experimentally the actual transfer 
function or functions. Several techniques for obtaining this information 
are available. The choice of technique depends upon the complexity of 
the system, available instrumentation, available evaluation equipment 
such as high response drives, and the accuracy desired. In general the 
accuracy and response of the instrumentation should be several times 
better than the desired accuracy of the measured transfer function. 

Use of Steady-State Data. Complex systems are more difficult to 
measure, and it is sometimes desirable to perform some analytical work 
before beginning experimental measurements. For example, an nth order 
linear system can, in general, be broken up into n first order equations. The 
individual equations can then be evaluated by obtaining steady-state and 
transient data for each. In breaking the original equation one must be 
careful to obtain physically significant combinations so that the steady- 
state and transient response can actually be measured. This technique is 
particularly valuable when the system has numerous inputs. For example, 
an nth order linear system with m inputs can be simplified to n first order 
linear partial differential equations. To obtain the m times n transfer 
functions would then require m times n pieces of steady-state data but 
still only n pieces of transient data. For a more detailed treatment see 
Ref. 4. 

The assumption that the output to input relationship is the same during 
transients as the known steady-state relationship is called the quasi-static 
assumption. This is particularly useful for highly nonlinear phenomena 
such as turbojet compressor maps or aerodynamic shock systems. See 
Ref. 9 and Chap. 25, Sect. 2. 

Transient Data. When suitable evaluation equipment such as sinus- 
oidal input generators and phase measuring equipment is lacking, the 
system can be excited by any available time varying signal such as a unit 
impulse, step function, ramp, or combination of these and the output 
compared with analytically derived response of many systems to the same 
input. Examples of this method are given in Sect. 2, Figs. 16 to 19 and 
Tables 12 to 14. Other examples are in Refs. 5 and 6. Methods of ob- 
taining the frequency response directly from the transient response to a 
step input are given in Chap. 22. 

A graphical method to estimate system time constants from step response 
transient data requires plotting the response (x) on semilog paper with 
time on the linear scale and magnitude on the log scale normalized so 
that the response x — > as t — > <*>. A system with a single time constant 









Table 7. Power 


No. 


Schematic 
Representation 


Type 


"Gate" 
Element 


1 


/A'r^ L 


Contact 


Contact 



\Hll|lh 

- jf &G P J 



Relay 



Contact 



— — , Generator 

Motor 



cn 



*J 



Generator 



Electronic 
tube 



Field 



Grid 



generator 



■MM; 



51 



W! 



Saturable 
reactor 



D-c coil 



G f^ 


-=rti 




5 A 






ijo 3 w 


--* f 



Silverstat 



Contacts 




Pump, P 



■fj Valve, 



Ui 



Valve 



Valve 
gate 



Fuel or heat L 
storage, P , 



Throttle 



Throttle 
valve 



p } 


f "£fW 


Rotating 


J#" 


power 


=nyr 


source 







Clutch 



Clutch 



Clutch 
disk 
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Amplifiers (Ref. 11) 



Typical 


Typical 


Approximate 


Devices 




Input 


Output 


Power Amplifi- 


Represented 


Power 


Units 


Units 


cation Factor 


by Load, L 


Control 


Ounces 


Watts 


1 X io 7 X * 


Relay motor, 
generator 
field, im- 
pedance, 
solenoid 


On-off 


Watts 


Watts or 
kilowatts 


1 X 10 3 


Relay motor, 
generator 
field, im- 
pedance, 
solenoid 


On-off 


Watts 


Watts or 
kilowatts 


50 


Motor, im- 
pedance 


Continuous 


Micro- 


Watts 


1 X 10 5 


Relay motor, 


Continuous 


watts 






generator 
field, im- 
pedance 




Milli- 


Watts 


3X 10 2 


Generator 


Continuous 


watts 






field, im- 
pedance 




Grams 


Watts 


1 X 10 7 X t 


Generator 
field, im- 
pedance 


Stepped 


Inch- 


Horsepower 


1 X io 7 X t 


Turret, 


Continuous 


pound 






press, 
heat ab- 
sorption 




Inch- 


Horsepower 


1 X 10 7 X * 


Propeller, 


Continuous 


pound 






vehicle, 

generator, 

mill 




Inch- 


Horsepower 


5 X IO 4 X t 


Vehicle, 


On-off 


pound 






mill 
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Table 8. Ekrob Correctors (Ref. 11) 



Schematic 
No. Representation 



zr~ 




itP 



Ml 



c 




Type 

Electric 

motor 


Input 
Energy 
Electric 


Output 

Energy 

Mechanical 

rotation 


Approximate 

Output 

Power Range 

1 X 10~ 2 to 4 

X 10 4 hp 


Solenoid 


Electric 


Mechanical 
transla- 
tion 


1 X IO" 3 to 15 
hp 


Hydraulic 
motor 


Hydraulic 


Mechanical 
rotation 


1 X 10~ 2 to 11 
X 10 4 hp 


Piston 


Hydraulic 


Mechanical 
transla- 
tion 


1 X IO" 3 to 1 
X IO 3 hp 


Steam or 
gap 
prime 
mover 


Heat or 
chemical 
(fuel) 


Mechanical 
rotation 


5 to 1.65 
X IO 5 hp 


Burner 


Chemical 
(fuel) 


Heat 


1 X 10 2 to 1:5 
X IO 8 Btu/hr 



T will give a linear response and the slope of this line is the reciprocal of 
the time constant, i.e., 

A log x k 



At 



T 



Note. For log e , k = 1, for logio, k = 0.434. A plot of the response of a 
system with two time constants (Ti > T 2 ) will start flat at log x (x = 
initial value) and then asymptotically approach the slope of the reciprocal 
of the larger time constant (k/Tx). Drawing the asymptote will give an 
intersection 



log 



x 



Ti + T 2 \ 



on the response axis. See Ref. 6. 
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Frequency Response. The most generally useful method is to excite 
the system or element under test with a sinusoidal signal. The frequency 
response is obtained by making a comparison of the amplitude and phase 
relations of the input and output over the frequency range of interest. 

The phase and amplitude relations can be obtained in a number of ways, 
e.g., from (a) direct oscillograph or recorder readings of the variables, 
(b) Lissajous patterns on a long persistence oscilloscope, and (c) special 
test equipment that gives a direct reading of the phase and amplitude 
ratios. 

An analytical expression approximating the transfer function can be 
obtained by curve matching techniques. Sufficiently good results are often 
obtained by a simple trial and error approximation of the frequency 
response obtained by the use of the straight line asymptote defined in 
Chap. 21. Straight lines with slopes which are multiples of 20 db per 
decade are first drawn so as to approximate the experimental data. The 
exact frequency response corresponding to the estimated straight line 
response is then calculated by use of the graphs of Chap. 21. The agree- 
ment between the calculated and measured response is checked and the 
process is repeated if necessary. With a little experience one or two 
iterations are usually sufficient. The intersections of the straight lines 
are the poles and zeros of the transfer function. More elaborate approxima- 
tion methods are available if needed. See Refs. 6 and 8. 

Correlation Technique. The autocorrelation function of white noise 
is an impulse. Therefore the cross correlation of the input and output of 
the system is simply the impulse response of the system when the input is 
white noise. An experimental setup similar to Fig. 10 can therefore be 
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Fig. 10. Test configuration to obtain system response by correlation technique. 

used to evaluate the transfer function of one element or system. Because 
the cross correlation filters all signals not correlated with the input white 
noise, the technique has the potential advantage of allowing the normal 
system operation to continue while the test is being conducted. See 
Ref. 8. The practical difficulties of mechanizing a satisfactory cross- 
correlator has limited the usefulness of this method. 
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2. CLASSICAL METHODS OF ANALYSIS 

System Equations 

In Methods of System Analysis in Sect. 1, the correct description of the 
system or component dynamic performance by differential equations was 
stated to be basic to all methods of analysis and synthesis. 

General Linear Differential Equations. A common type of system 
equation, the general linear integro-differential equation with constant 
coefficients may be written in terms of the input x(t) and the driving 
function y(t) (Ref. 13) as 

d n x{t) d n ~ l x{t) dx{t) 

—rz — \- «i — r— ; 1 V a n -i — — 

dt n dt n ~ l dt 



(22) a —^ r + «i , w-1 H H a n -\ -—- + a n x(t) 



+ fln+i I x(t) dt-\ \- a n+q j x(t) dt q = y{t). 

As a class, the homogeneous equation resulting from reducing the right- 
hand side of the equation to zero has as its general solution a linear combina- 
tion of solutions of the exponential form e Pnt , where p n may be real or 
complex. 

Characteristic Equation. The operator p = d/dt together with 

1/p = I dt may be substituted into the reduced homogeneous equation. 

The resulting operational equation may be handled by the rules of algebra 
as explained in Chap. 8, Sect. 1. Factoring out the operational part of 
this equation yields the characteristic equation (Ref. 13) : 

(23) a oV n+q + a l v n+q - 1 +-'- + 

a n _i2? 1+5 + a n p q + a n+1 p q ~ l -\ \- a n+q = 0. 

General Solution to Linear Differential Equations 

The complementary solution to eq. (22) is 

(24) x t = An+ge^n^t + A n+g _ ie (p« + *-i>< + • • . + A^ 1 , 

where p n are the roots of the characteristic eq. (23). The complete solution 
is (Ref. 13) 

(25) x(t) = x t + x s , 

where x s is the particular solution to eq. (22). The particular solution is 
obtained by substituting an assumed solution and solving for the coef- 
ficients. (See Part A, General Mathematics.) 
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Absolute Stability Defined. (See also Chap. 21.) The stability of a 
system may be broadly defined as that property which insures that it 
will remain in operating equilibrium through normal conditions (Ref. 14). 
A system is said to be on the verge of stability when it is hunting, that is, 
subject to sustained oscillations; if the oscillations grow, the system is 
unstable; if they decay, the system is stable. Nonoscillatory instability 
is also possible, such as the exponential growth of a system variable in 
response to a disturbance. Tables 14 and 15 give examples of stable and 
unstable performance and the dependence of stability upon the nature of 
the roots of the characteristic equation (or exponents of the complementary 
solution). When system gain is increased to provide desired accuracy, 
instability is frequently encountered. This is the situation which is 
attacked with equalization (or stabilization) methods designed to provide 
a margin of stability without compromising system accuracy. A margin 
of stability is nearly always desired from the hunting condition. It is 
implied that the system is linear or may be linearized in the neighborhood 
of the operating point for the purpose of analyzing stability (for lineariza- 
tion of nonlinear systems, see Linearization, Chap. 25, Sect. 2). 

Example. Second Order System (Motor Synchronizing on a Fixed Signal). 
In Fig. 11 a motor drives a load to which it is coupled directly from an 



Final position 



Initial position 
of controlled 
_ variable = c 

Torque 
input = - Kcj 

Motor 

Fig. 11. Motor driving load from an initial position Co to correspondence at position 0. 
Combined inertia = J lb-ft-sec 2 ; damping = D lb-ft/rad/sec; stiffness = K lb-ft/rad 

(Ref. 3a). 

initial rest position c to correspondence at position (Ref. 13). The 
input to the motor produces a torque that is proportional to the differ- 
ence between the controlled load position and the reference input position. 
Thus motor torque equals — Kc since the desired final position is zero. 

Present in the load and the motor are mechanical friction and electrical 
damping torques both of which are proportional to the motor speed; 
friction and damping torque equal D(dc/dt). Static friction forces are 
negligible. There is also a torque due to the combined inertia J and this 
torque equals J(d 2 c/dt 2 ). 
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The complete torque equation can now be written as 
d 2 c dc 

dt 2 Jt 



(26) /— + !> — + Kc = 0. 

\ ' -.,2 ■■• 



The steady-state displacement is zero, that is, the corresponding position, 
so that the transient response is the entire motion. By writing the charac- 
teristic equation as 

o D K 

(27) p 2 + _ p + _ = 0> 

a further modification will be made in the interest of obtaining a simpler 
form of the solution. Let 

Ik 

(28) /— = w = undamped natural frequency, 

and 

D 

(29) — /= = f = damping factor. 
2 v KJ 

If these substitutions are made, the characteristic equation may now be 
written (in nondimensional form) as 

(30) p 2 + 2f«oP + coo 2 = 0, 
in which the two roots are 



(3i) vi = -[r-Vr 2 -i]co , 



(32) p 2 = -[f +Vf 2 - IK. 

The effect of f upon the form of the transient solution of a second order 
system is treated in the next section. 

Use of Laplace Transform 

The work involved in using the classical approach to the solution of 
linear differential equations may be simplified to a routine process through 
the use of the Laplace transform and its inverse, which uses the same 
approach to obtain both transient and steady-state solutions (Refs. 15, 
16). The Laplace transform has the advantage of handling initial condi- 
tions and discontinuous inputs directly. For a complete presentation of 
this method see Ref. 17. 

The Laplace transform is defined as 

4/(0] = F(s) = I f{t)e- st dt 



Jo 
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The inverse Laplace transform is defined as 

_ 0. 

2ttj " 



£-i[F( s) ] = f( t ) =^-f F(s)e ts ds, t ^ 



In these definitions s is the complex operator a -\- jco. The abscissa of 
absolute convergence, denoted by a , is at <r > 0. 

The Laplace transform and inverse Laplace transform are referred to 
as a transform pair. 

Laplace Transform Applied to Feedback Control System. A 
closed loop linear control system may be represented in terms of the com- 
plex operator s by the eqs. (33), (35), and (36) as follows: 

(33) C(s) = G(s)E(s), 

where C(s) = transform of the controlled variable, c(t), 
E{s) = transform of the actuating error, e(t), 
G(s) = transform of the transfer function of the forward control 
elements and may be given the factored form : 

K(S — So) (S — Sb) ■ • ' 

(34) G(8) = 



s n (s - s^){s - s 2 ) ••• 

(35) E(s) = R(s) - B(s), 

where R(s) = transform of the reference input, r(t), 
B(s) = transform of the feedback, b(t). 

(36) B(s) = H(s)C(s), 

where H(s) is the transform of the transfer function of the feedback ele- 
ments and may be similar in form to that given in eq. (34). 

The block diagram for the above system of equations is given in Fig. 
256. The transform of the closed loop transfer function (see Fig. 25c for 
the block diagram) for this control system is 

C(s) G(s) 

(37) 



R(s) 1 + G(s)H(s) 
Expressing C(s) in terms of polynomials in s, 

N(s) a v s v + a t ,_is v_1 -\ \- ais -f- a 



(38) C(s) = 



D(s) s« + b^s**- 1 + • • • + M + & 



where, because of the nature of the functions G(s), H(s), and R(s), in 
general q ^ v. 



Table 9. Some Useful Laplace Teansfobm Paies (Ref. 15) 

No. F(s) f(t), O^t 

1 - 1 or u{t), unit step at t = 
s 

2 - e~ as u(t - a) 
s 

3 - (e~ as - e~ bs ) u(t - a) - u(t - b), a<b 
s 

a 1 v u tt) — u(t — a) ... . 

4 1 hm . unit impulse at t = 

a-»o a 

5 ~2 ^> un it ramp at £ = 

R _ /n-l 

s n (n-l)! 
1 



1 




s + a 


8 




1 


(« + a) 2 


9 




1 


(s + a) n 


10 




1 


s(s + a) 


11 




1 


s 2 (s + a) 


12 




1 


s(s + a) 2 


13 




1 


(« 


+ a)(s + 7) 


14 




1 


1 f» 


s(s 


: + a)(s + 7) 

1 




s 2 +j8 2 


16 




s 


s 2 +/3 2 


17 




ais + ao 



+ ( 



te~ at 




tn—\~—at 

(n-i)r e 




- (1 - e~ al ) 
a 




e~ at + <xt- \ 




a 2 




-= (1 - e _a ' - ate" 


-an 



-a< 



e-i" 



7 — a 

CH7 0:7(0; — 7) 

1 • „ 

- sin p£ 



coSjSi 



A cos 08* + 1/0 
A = [a, 2 + ao 2 //3 2 ]* 

ai 
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Table 9. Some Useful Laplace Transform Pairs (Ref. 15) (Continued) 



No 




m 




/(0. o ^ t 


IS 




i 




^pism fit-fit cos 0t) 


(s 2 + /3 2 ) 2 


19 




s 




-sinpt 


(s 2 + /3 2 ) 2 


20 




s 2 




— (sinjSf + pt cos (3t) 


(s 2 4- /3 2 ) 2 


21 




s 2 -/3 2 




t cos jSi 


(s 2 4- )8 2 ) 2 


22 




1 




-e- a 'sin/3f 


(s4-a) 2 4- / 3 2 


23 




s 4- oc 




e - ™' cos /3f 


(s + a) 2 4-/3 2 


24 




ais 4- oo 




Ae- a< cos (fit 4- i/0 


(s 4- a) 2 4- 2 










A = [o^ + Caia-ao) 2 ^ 2 ]^ 


25 




1 




^tan- l(aia ~ ao)//3 
ai 

- sinh jSi 


s 2 -/3 2 


26 

27 




s 




cosh /3i 


5 2 -/8 2 

S F(s)-/(0+) 


28 


s 2 F(s) 


- S /<o + )-f> 


(04-) 


d 2 m 

dt 2 


29 




*•(«) , / ( - 1} (04-) 
s s 




ff(t)dt 


30 


F(s), 

s 2 " t " 


/ ( - 1) (04-) , / ( " 2) (04-) 

2 ' 


j[Jf(t) dt] dt 


31 




aF(«) 




af(t) 


32 




Fi(s) ± 7? 2 (s) 




/i(0±A(0 


33 




a^(as) 




/O 


34 




^C* + a) 




6- B '/(0 


35 




F(s - a) 




e at f(t) 


36 




e* as F(s) 




f(t =F a), o > 
where f(J — a) = 0, < £ < a 
/(« + a) = 0, -a < * < 
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The inverse transformation £~ 1 [C(s)], which is the solution c(t), may be 
obtained for the simpler cases by direct reference to transform Tables 9, 
10, and 11 or Tables 1 and 2, Chap. 9 (see also Ref. 15). However, in 
general it is necessary to express C(s) as a sum of partial factions with 
constant coefficients, whose inverse transforms may then be readily ob- 
tained from the table of transforms and summed to yield the complete 
solution, c(t). 

Table 10. Laplace Transform Pairs 

for First Order System X(s) = F(s) 

l s ~p i 

Input Response 
as Function Time Response 

Input as f(t) F(s) as X(s) as x(t) 

Impulse, A\hm U{t) - u{t - a) l 
La-o a J 

whence A = f^M) dt A ^T7 ^ e -" r 

Step, MO f ^^ ,1(1-.-./*). 

Ramp,il« £ g2{T f + 1} At-AT(l-e-l*) 

Partial Fraction Expansions of the Laplace Transform. For ex- 
pansion into partial fractions the denominator of the rational proper 
fraction C(s) is first factored, yielding (Ref. 16): 

N(s) N(s) 

(39) C(s) = 



D(s) (s - s0(s - s 2 ) • • • (s - s k ) • • • (s - s n ) 
Expansion into partial fractions with first order poles only (Ref. 13) 



D(s) s — s 1 s — s 2 s — s k s — s r 

leads to the solution (see transform pair No. 0.11 of Ref. 15) 

n N(si.) 
(41) c(t)= Z-7^^, 

fc =i D'(s k ) 

where 

a T d 
-D(s) 
.as 



(42) D'(s k ) = 



( = means equal by definition) . 



2 

Table 11. Time Responses of Second Order System C(s) = „ , -„ ; o-R(s) 

s z + 2foj s + oxf 

Type of Input 

£[r(t)] = B(s) f < 1 r = 1 " f > 1 

Unit impulse, — e~ at sin 8t aHe~ at -^— (e~ at - e~^) ~Z- 



y — a 



O 



. u{t) - u(t - a)-] , K > 



["lini } ~ u{t 1 = 
L a->o a J 

Unit step, 1 + — e~ at sin (pt - tan" 1 — ) 1 - (1 + at)e~ at 1 + 

6 \ —a/ 



ye 



-at _ np -yt 



> 

1 « - 7 £ 

£[w(0] = - O 



s 



Unit ramp, * 5 + - e~ at sin ( 0t - 2 tan" 1 — ) t - - + te~ at + - e~ at t- 1 +- — -< 

60 8 \ —a/ a a ay ay{y — a) ;£j 

m = \ - £ 



Specific form 
of C(s)/R(s) 



ay 



(s + «) 2 +/3 2 (s+a) 2 (s+a)(s+y) > 

where a = fwo where a = wo where a + y = 2fcoo c/5 

and /3o 2 = a 2 + /3 2 = wo 2 and ay = coo 2 c75 



Note. Laplace transform, C(s), of each time response in this table is simply the product of the transform of the input, R(s), by the sys- 
tem transform C(s)/R(s) in the table. 



to 
o 

CO 

in 
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The general case of expansion into partial fractions with higher order 
poles yields the solution (see transform pair No. 0.21 of Ref. 15) 



f"* _i e Sfc *, mi -\- m 2 -\ \- m n — q, 



n m k rz 

(43) c(0= EEt^ ,, 

fc=i i=i (rn k - i) ! 
where 

A i r^ -1 (s-s k r*N(s) 

(44) K ki = 



and 



(i- l)\lds 



J-i 



D(s) 



(45) D(s) = (s - Sl ) w i (s - s 2 ) TO « • • • (s - s fc ) m * •••(»- s n ) m ». 

Order of System Responses as Seen from Partial Fraction Expan- 
sions. Any complex linear system can be represented as a combination of 
first and second order systems. This may be seen from the partial fraction 
expansion of a system response function such as that of eq. (38) into 
partial fractions 



(46) C(s) = 



N(s) 
D(s) 

s — Si 



+ 



C 9 



s 2 



+ ••• + 



C» 



+ 



axs + a 



s - s k s' + 2fco s + aV 



+ 



where two conjugate complex first order poles have been combined into a 
single term. 

The response of the complex system can, therefore, be considered as 
the sum of the responses of first and second order systems. The responses 
of first and second order systems are thus of considerable importance and 
are given for various inputs in the following. 

First Order System Responses. A first order system is characterized 
by a single energy storage. An example of a first order system is the 
simple hydraulic servo of Table 5 for phase lag. The system equation is 
thus (using T for T v ), 



(47) 



dx(t) 
T-j- + x{t) = {b/a)y{t) 
at 



fit). 



The transform of the equation may be written: 

1 



(48) 



[X(s)] = 



Ts+ lj 



im] 



which is of the form (see Ref. 16) 

(Response function) = (System function) (Excitation function). 



£ 100% 



(a) .S2 

° 63.2% 




Final value 



Fig. 12a. Response of first order system to step showing time constant relationships. 
1.0 
0.9 
0.8 
0.7 



(b) »N 
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<- Impulse, — ;- 
A 























































































































































Fig. 12b. Step and impulse response of first order system: 



T^ + x=f(t). 
at 



xT r°° 

Impulse: — = e~ t,T . A = I (impulse function) dt. 

A J-oo 

Step: — = 1 — e~ i,T , A = magnitude of step. 

A 
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The characteristic equation is 

(49) Ts + 1 = 0, 
and the transient solution is 

(50) x t = A 1 e~ tlT . 

The performance can be characterized by the quantity T, called the 
time constant of the system (see Ref. 13). Physically, the time constant is 
the time to complete 1 — e _1 = 63.2% of the change after either a step 
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12 3 4 5 

J_ 

T 
Fig. 12c. Ramp response of first order system: 

dx 

T— + x=f(t) =At, 
at 

JL = L _ (i _ e -tlT^ 
AT T K ' 

or impulse input. Also it is the time given by the intersection of the 
tangent to the transient at t = with the asymptote to the final value 
when a step or impulse is applied at t = (see Fig. 12a). 

Table 10 lists three types of input f(t), the corresponding excitation 
function F(s), the response function X(s) for the system function 
1/(T S + 1) and the inverse transform x(t) of the response function, X(s) 
and x(t) forming a Laplace transform pair. In Fig. 126 are plotted the step 
and impulse response of a first order system obtained from the solutions 
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appearing in Table 10. In Fig. 12c is plotted the ramp response of a first 
order system from the solution appearing in the same table. 

Second Order System Responses. The solutions for unit impulse, 
step and ramp inputs to the second order system of eq. (26) generalized 
by setting the right-hand side equal to Kr(t), namely 



(51) 



d 2 c dc 

J — +D- + Kc = Kr(t) 
dt 2 dt 



are illustrated respectively in Figs. 13, 14, and 15a, 6, c. The transform 
0.8 



cm 

w 




Fig. 13. Response of second order system: 

C(s) 



s 2 + 2fco s + wo 2 
to unit impulse in r(t) for various values of f . 



R(s) 
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Fig. 14. Response of second order system: 



C(s) = 



s 2 + 2fw s + wo 2 
to unit step in r(t) for various values of f . 



m 




•Asymptotes intercepts at 2f u t 
(a) 




Fig. 15a, b. Response of second order system: C(s) = [(a>o 2 )/(s 2 + 2f«os + uo 2 )]R(s) to 
unit ramp r(t) = t 2 for various values of f. 
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Fig. 15c. Response of second order system: 



C(s) = 



S 2 + 2j"co S + coo 2 

to unit ramp r(t) = t for various values of f . 



m 



of this equation may be written in a nondimensional form (as for the 
first order system of eq. 47). 

(Response function) = (System function) (Excitation function) 

(see Ref. 16), which in this case is 



(52) 



C(«) = 



O>o 



s 2 + 2fa s + w 2 



[R(s)] 



The time response may then be obtained by looking up the inverse 
transform pair in a table of Laplace transforms (see Ref. 15). The form 
of the response will be oscillatory, critically damped, or overdamped as 
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the damping factor f < 1, =1, or >1. Table 11 is a chart of the time 
responses illustrated in Figs. 13 to 15 (see Refs. 6, 13, 18, and 19). 

Tables 12 and 13 and Figs. 16 to 19, from Ref. 20, are for the determina- 
tion of equation coefficients and system parameters for second order sys- 
tems. Table 14 illustrates time responses. Table 15 treats stability. 
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h = transient overshoot from steady state 
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Fig. 16. Determination of equation coefficients for second order systems from response 

curves (Ref. 20). 



p 



Table 12. Relationship among System Parameters and Equation Coefficients for Second Order System (Ref. 20) 



a 2 [(d 2 x)/(dt 2 )] + a 1 [(dx)/(dt)] + a^x = y(t) 







Definition in 






Terms of Equati 


Parameter 


Symbol 


Coefficients 


Damping ratio 


f 


«1 


2\Zaoa,2 


Undamped angular natural 
frequency- 


co 


/flo 



Undamped natural frequency 



Undamped natural period 



Angular natural frequency 



Natural frequency 



To 



At 



2tt 



2^p 



jao _ / «i\ 2 
\a 2 \2a 2 ) 

2tt \a, \2a 2 / 



Equivalent Expressions 

1 T T l + r 2 y+1 
woT 7 / 2irr e ' 2-V/W2* 2\/^ 



co 



„ , 2tt 1 
27r/o,7ir'r^r 



1 



i i 



co 



2tt 2t\ZT^T* T 2tt{T c 



wo /o 



t 7 rn To 

qjp-y/l - f 2 _w J_ Vl - f 2 
2tt ' 2tt' T ' 27rfT c 



a 

CO 

> 

n 

n 
O 

z 

—i 

SO 

o 



Natural period T 



Critical time constant 



2tt 2tt 2tt 2tt{T c 1 T 



\ a 2 \2a 2 / 



a 2 \^a 2 / 

2o2 J_ _To_ Ty/\ - r 2 2TiT 2 2Ti Tj_ 

a x fwo' 27rf ' 2xf ' Ti + T 2 ' v + 1* 2f 2 



Large time constant (f > 1) ^ / n — - )===. - — - ^^. VtfTe, vT 2 , ^- 2 

wolf — \/f — 1] «o 2tt y + 1 z 



01 

2a 2 


1 


a 
a 2 



2a 2 + V \2aJ 



Overcritical time constant T t — — . Tx + T*. ^— ^ Tx, 2f 2 7 7 c , -^= T 7 , 

«o coo v -\Jy 



'V 



\(£) 2 - 



O 



Small time constant (f > 1) r 2 ;=^== *. — !=-' — ^>=' S. — | 

o wo[f + -y/T 2 — 1] -y/vwo 2tt^/v -y/V v £! 



a 2 c/> 

O 






_ai_ , -//_oi_y_ oo ^ 

t,. f + - ,,. . n 2a 2 + \ V2a 2 / a 2 f + V {* - 1 Tx . > 

Time parameter ratio (J > 1) v , > — £L 



2a 2 \ \2o2/ a 2 -< 

to 



p 



Table 13. Determination of Equation Coefficients for Second Order Systems from Response Curves (Ref. 20) 

Equation Coefficients 

in Terms of ao and 
Equation Parameters 



Transfer 
Function 



K-j+- s+1 ] 

L \coo/ WQ J 



Type of 
Response 



Oscillatory 
< f < 0.5 



[(£> 



2r i 

+ -S + 1 
WQ J 



L \wo/ "o J 



Equation 

Parameters Method Used to Find 
Response Curve Used Equation Parameters 



x can be any peak 



i ° iyw - 

x can be any peak 

t 



x g can be any peak 



f , T Measure T, 

XQ, Xi, x 2 , x Sj 



Form ratios 

Xi X 2 X S X 2 X 3 X 3 
— ) — ) — ) — ) — > — 
X X X Xl Xl X2 



Find r from Fig. 17 



IT 4:ir' 



o 

O 



a 

> 

n 

7\ 

n 
O 

z 

TO 

o 



[( i ) + -« +1 J 

L \W0/ WQ J 



Near critically 
aperiodic 

0.5 < f <2.0 



1 



[(£> 



2 2f n 

+ — s + lj 



WQ/ WQ 




'l *2 '3 



f, wo Measure <i, <2> <3 
Form ratios 

h h tz — t 2 
— j — , 

h h h — h 

Find f , wo<i, cooh, woh 

from Fig. 18 
Compute value of 

f = fav and coo = «0av 



2oof 

wo 



do 
wo 2 



Critically 



[ (— \ + — s + 1 1 aperiodic 

L\co / coo J . „ 

f = 1.0 



[( £ ) + - s+1 ] 

L \wo/ con J 



0.736 
0.264 —pi 



1.00 
0.736 



0.264 




Measure T c on Fig. 18 

for r = i.o, 

T c = h = fa — h = h — fa 



2a T c 



s 



Nonoscillatory 



r(-) +— s+ii r>1 . 

L Vcon/ con J 



•[(-)'+-• + 1 ] 

L \coo/ coo J 




Semilog plot of re- 
sponse curve 




Invert response curve 
to agree with above 
plot, then plot on 
semilog paper. 



vi, T\ Plot response curve on 

semilog paper. Extrap- 
olate straight line por- 
tion of plot to t = 0. 

Measure In x\ and In xi at t\ 
and fa respectively. 

Measure xo and X( eX ) 

Compute T\ from 



Ti = 



fa-h 



In x\ — In X2 



Compute v from 

Z(ex) 



Z(ex) — zo 



ooTid' + 1) 



> 

m 
Z 
— i 
> 

■ — 

o 



■< 
en 



> 

Z 
> 

en 



O 



Step function 
position 



© 



Table 14. Time Responses of Some Common Transient Modes (Ref. 20) 

F(s) f(t) Time Response 

f(0 



1 



to 
o 

oo 



Step function 
velocity 



Step function 
acceleration 



© 



© 



First order lag 
converging 



T c s + 1 



— ,-tlT. 



f(t) 



f(t) 




D 

CO 

> 

n 

o 
O 

z 

—I 

o 



© 



s(T c s + 1) 



1 - e~ tlT o 



First order lag © i 

diverging 



TcS+1 






no 
i/r c 



f = 



Converging 
second order 



Diverging 
second order 

r= in 



© 



s(-T c s + l) 



Undamped (f) 

second order 1 



(s 2 /W) + 1 



r = o 



(4 + 2^ S+ l) 



® 



\C0(f OJ / 



1 






.l + e «/r c 




coo sin cooi 



/w 



1 - r — -^ sin (^cooVl - & + tan- 1 v ) 



1 - ~7r^F sin v 00 ^ 1 " r ' 2/ + tan-1 k' ) 




> 

m 

Z 
—i 
> 

en 
O 



en 

-< 



> 

Z 
> 

to 



O 
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x m - X\ 




•Xm — XQ 



*m = X2 
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Fig. 17. Damping ratio of oscillatory transients as a function of subsidence ratio for 
second order system (Ref. 20) of the type: 
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1.000 
0.736 

0.406 
0.199 



rU- 




tl <2 <3 



KZ& ) + I S+ " 



1.000 
0.801 
0.594 
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responce to step function 
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Fig. 18. Chart for determining f and wo for second order system when f is close to 1 

(Ref. 20). (The calculations on which these curves are based were made by Prof. John D. 

Trimmer of the University of Tennessee.) 
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Fig. 19. Time constant ratio Ti/T 2 as a function of damping ratio for overdamped 
second order system (Ref. 20). 
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Table 15. Stability As a Function of the Nature of the Roots of the 



Characteristic Equation (Ref. 20) 



Type of 
Stability 
of System 

Stable 
Stable 

Verge of stability; un- 
damped oscillatory 
time response 

Unstable 

Unstable 



Nature of Roots of 
Characteristic Equation 

(or Exponents of 
Complementary Solution) 

All roots have negative real parts. 

A single zero root; all other roots, if any, 
have negative real parts. 

Conjugate imaginary roots all different, 
in addition to roots for stable systems 
above, if any. 

Roots with positive real parts, in addi- 
tion to other types of roots, if any. 

Repeated zero or conjugate imaginary 
roots, in addition to other types of 
roots, if any. 



No. of Example 
of Performance 
Given in Table 14 

Nonoscil- Oscil- 
latory latory 



9 



10 



Application of Convolution Integral 

A convenient method of calculating the time response of a system to 
any arbitrary input makes use of the convolution integral (see Ref. 21), 
which may be written as 



(53) 



#) = f f(r)g(t - r) dr, 

J rr, 



where c(t) is the time response, f(t) is the input, and g(t) is the weighting 
function or characteristic time response to a unit impulse (see Weighting 
Function in Chap. 9). To evaluate this equation the arbitrary input is 
approximated by a series of impulses as shown in Fig. 20. If the im- 




Fig. 20. Approximation of a function, f(t), by a series of impulses. 
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pulsive response, g(t), is known, the sum of these responses to the impulses 
approximating the input signal constitutes the total time response, as 
illustrated by Figs. 1.47 and 1.48 of Ref. 8. Of course, a theoretical im- 
pulse function has zero width in time; in the practical case, if its width 
is much smaller than the response time of the system being considered, 
the results obtained will be valid. The quantity /(r) is the average height 
of the rectangular approximation of an impulse; At is the width; and r 
is the time to the center of the rectangle as illustrated in Fig. 20. 
The value of the time response at time t± may be expressed as 

(54) c{h)= 2 /MATflrfo-r). 

T = T1,T2, ' ' ',tl 

This indicates that c(t{) is the sum of responses to impulse inputs, all 
evaluated at t\. 

This same method may be used with the transfer function of the system, 
since it is simply the Laplace transform of the weighting function. 

Steady-State Solution of System Equations 

Although the complete solution of a linear differential equation for a 
system subjected to some driving function contains both transient and 
steady-state portions, the steady-state part can be obtained independently 
of the transient. 

Sinusoidal Driving Functions. The general form of the steady-state 
response of linear systems to sinusoidal excitation is sinusoidal and of the 
same frequency as the driving function. An example of the steady-state 
response c s (t) of the second order system of eq. (51) to a sinusoidal input 
is given in Fig. 21. 

When steady-state excitation with sinusoidal driving forces is considered, 
the Laplace transform is intimately related to the impedance concept. 
For the Laplace transform it will be found that s may be replaced by jw 
to obtain the steady-state response to a sinusoidal driving function (see 
Ref. 22). In Table 16 are given typical terms of an integro-differential 
equation showing use of the operator jw to obtain the electrical and 
"motional" impedances of analogous electrical and mechanical forms. 

The justification for this substitution of joi for s is given in Ref. 22. 
Application of this technique to the differential eq. (52) yields: 

con 
0«) 2 + 2fco 0«) + "o 2 

Complex Plane Plot. The steady-state response of a system as a func- 
tion of frequency is very useful in servomechanism and regulator design 



FUNDAMENTALS OF SYSTEM ANALYSIS 
1.0 1- ^-v r(t) = sin wt 



20-55 



c 8 (t) = 



(o 2 sin (cct -tan" 1 2 ^° u ) 
-y/(u 2 - w 2 ; 2 + (2J-w w;2 




-1.0 L- 



Fig. 21. Example of steady-state response, c s (() to unit amplitude sine wave input, 

v(t), for second order system: 



C(s) 



s 2 + 2fw s + wo 2 



for f = 1 and w = 0.5«o. 



Table 16. Summary op Equation Terms and Complex Quantities (Ref. 3a) 



Physical 
System 

Electrical 



Derivative 
Form 
di(t) 

Ri(t) 



e/*> 



dt 



Mechanical 



M 



dv(t) 



K 



dt 
Dv(t) 



dt 



Transform 
Form 

Lsl(s) 

RI(s) 

17W 
C s 

MsV(s) 

DV(s) 
KV(s) 



Complex 
Form 

juLI{ju>) 
jo>MV(ja)) 

r W 

1to 



Complex 
Impedance 

juL = jX L 
R 

J = ~ jXc 

joiM 
D 



-J 



K 
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(Ref. 23). Use is made of complex plane diagrams in which the magnitude 
and the angle of the output to input ratio are shown by a single line on 
the complex plane as in Fig. 22. The complex output-input ratio C/R 
is obtained by substituting ju for s in the transform eq. (52). 




Imaginary 
-0.8/ 

Fig. 22. Complex plane plot of 

C 



R (» 2 + 2i-«oC/«) + "o 2 



for control system of Fig. 11. 



Logarithmic Plots. Instead of plotting vector loci of the transfer 
function as in Fig. 22, the contours can be plotted to a logarithmic scale 
(see Refs. 24, 25, and 26). To exploit certain manipulative advantages, 
the attenuation and phase angle graphs are made separately. The attenua- 
tion is plotted in decibels, or 20 logio | atten. | versus the logio co; the phase 
angle is also plotted versus logio w. In Fig. 23 the complex transfer func- 
tion of eq. (55) has its attenuation and phase angle plotted against the 
logio(w/a>o), giving a nondimensional chart for the frequency response of 
second order systems over a range of values of damping factor f . 

3. BLOCK DIAGRAMS 

Definition of Terms 

A block diagram is a simplified method of presenting the interconnections 
of significant variables. It displays the functional relationships rather 
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than the physical and thus gives a clear insight to the problem. The 
physical system and interrelationship determine the block diagram arrange- 
ment, each block is a logical step in the flow or signal process. Block 
diagrams are built up by algebraic combinations of individual blocks where 
each block is a transfer function. An example is shown by Fig. 24 where 
the transfer function of the controlled system is G 3 = C/M 5 . Most 
block diagrams only show the desired inputs and outputs; however, in 
many physical systems there are loading and regulating effects. These 
effects must be considered and can be handled as separate input effects. 
The recommended nomenclature (Ref. 29) for symbols in the block 
diagram is illustrated by Fig. 24 where: 

V = desired value, 
R = reference input, 
E = actuating error, 
M = manipulated variable, 
U = disturbance function, 
C = controlled variable, 
Q = indirectly controlled variable, 
B = feedback. 

The symbols used, such as C, may be in Laplace, operational, or sinusoidal 
form and can be indicated as C(s), C(p), or C(jco). Lower-case letters are 
used to indicate time functions (r, v, e, m, u, c, b). Generally the parentheses 
( s )> (p)> or 0' w ) are dropped unless a particular form of representation is 
required. 

The transfer functions are labeled as follows: A for reference input, G 
for forward elements, i.e., from error to output, N for disturbance input, 
Z for indirectly controlled system, and H for feedback. Numerical sub- 
scripts are used to identify individual elements. References 29 to 31 are 
the standards of the American Institute of Electrical Engineers and the 
Institute of Radio Engineers. The important point is that a consistent 
system be used. 

Construction and Signal Flow 

As illustrated by Fig. 24 the arrows connecting the blocks indicate the 
unidirectional signal flow. The circular junction point with appropriate 
plus or minus signs is used to indicate summing or differencing points 
respectively, i.e., 

x+y x ~s-^y — x 



5*^ -rrS^ 
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Fig. 23. Magnitude and phase shift of C/R versus frequency ratio w/wo for various values of f. 
Note. For f > 1 plots are simply those for two unequal time lags (Ref. 3a). 
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M 4 



O 



M K 



Controlled 

system 

G 3 



Feedback 

elements 

H 2 



Feedback 
elements 



Indirectly 
controlled 
system Z 



CO 

> 
n 

n 
O 

z 

■H 

o 



Fig. 24. Block diagram of representative closed loop system. 
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The I.R.E. standard graphical symbols may also be used (Refs. 30 and 
31). 



Mixing point: 



X\ •— v JC3 

H f ) — >■ *3 = f(xi,x 2 ) 



X2 



Summing point 

2 j >■ x 3 = xi- x 2 



+ 

*3 ± 

X 4 > 



x$ = xi-x 2 + x 3 -x 4 
■ *5 



Multiplication point: 



x i /~\ x 3 

>{ ir ) — — ->■ *3 = *1*2 



x 2 



Algebra of Block Diagrams 

A complex block diagram can be rearranged or reduced by combining 
blocks algebraically. When all the loops are concentric, the indicated 
manipulations can be carried out directly by successively applying the 
relation C/R = G/(l -\- GH) to the innermost loops. When the inner 
loops are not concentric or even intertwining, the block diagrams can 
usually be reduced to concentric loops by the following rules and by refer- 
ence to Table 17. 

1. Data takeoff channels can be moved forward (in the direction of 
arrows) or backward in the system at will except that the takeoff point 
cannot pass a summing point. Whenever a data takeoff branch is moved 
forward past a function G, the function 1/G, must be added in series with 
the branch. Whenever a data takeoff branch is moved backward past a 
function G, the function G, must be added in series with the branch. 

2. A channel feeding into a summing point can be moved forward and 
backward in the system at will except that it cannot pass a data takeoff 
point. As this feed channel is moved forward in the system past a func- 
tion G, the function G must be added in series with the channel. As it is 
moved backward past a function G, the function 1/G must be inserted in 
the channel. 

3. In some cases, it will be found necessary to move a takeoff point 
past a summing point or a summing point past a data takeoff point in order 
to reduce the system block diagram to simple concentric loops or parallel 
paths, which can be handled by methods (1) and (2). This can be done 
by removing a troublesome feedback point or data takeoff by closing an 
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Table 17. Theorems for the Transformation and Reduction of Block 
Diagram Networks (Ref. 17) 



Theorem 



Original Network 



Equivalent Network 



1. Interchange 
of elements 



K,G, 



KoGo 



^2^2 




K& 


b y 







2. Interchange of 
summing points 



a±b^ ^\a±b±c 



-°-^_Q a± b> Qa±b$i 



a±c /~^a±c±J} 



^vnt 



3. Rearrangement 
of summing 
points 



a±6±c 

I >■ 



{b±c 



a /-^ ai-b / ~^a±b±c 

-»-Q ? — *- 



'), 



4. Interchange of 
takeoff points 



K,G, 



6-<r- 



^2<J2 ' > 



K,G, 



6^- 



K2G2 



-?-& 



^*~b 



5. Moving a 
summing point 
ahead of an 
element 



d = b±c 



a -» r 


~\ a±c'^ 


^G, 


V 


c '> 


» 








1 






K x d 



6. Moving a 
summing point 
beyond an 
element 



a±b 



a ss si 



KiG, 



c = a'±6' 
6' 



KiG, 



7. Moving a 
takeoff point 
ahead of an 
element 



tflGj 



8. Moving a 
takeoff point 
beyond an 
element 



k,g. 







XiCj 


6 














1 




K1G1 




" 





9. Moving a 
takeoff point 
ahead of a 
summing point 



o 



c = a±b 



M6 



0^> 

c = o±6 



10. Moving a 
takeoff point 
beyond a 
summing point 



c = a±6 



■o 



a = c*b Jb 6 A 



c = o±6 



11. Combining 
cascade 
elements 



if,Gj — >— iC 2 G2 



(X-jG^^z^) 
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Table 17. Theorems for the Transformation and Reduction of Block 
Diagram Networks (Continued) 



12. Removing an 
element from 
a forward loop 



^-1 K lGl |— ^-Q— ^ 

\d = b±c 



Jf 2 G 2 



KoG' 



-rxrV^^iG 



K 9 G. 



13. Inserting an 
element in a 
forward loop 



K1G1 



■^O 



d = b±a 



jf-Q -rK 2 G 2 > K& -*Q-^ 



KfG; 



d = b±a 



14. Eliminating a 
forward loop 



*iGl 


6 /" 




C 


#2 G 2 







d = 6±c 



* K 1 G 1 ±if 2 G2 ;iL -> 



15. Removing an 
element from a 
feedback loop 



6 r = n-l 



KnGl 



1 

#2 G 2 


6' ' ' 


^K lGl -i 




, 


c' = a'±6' 



16. Inserting an 
element in a 
feedback loop 



^-c 


■> c ^« 


X1O1 




d, 


J > 






b< 


^ c = a + 6 









^2 G 2 


S 


> 


K 7 


1 
G 2 


9'K 1 G 1 i 


6' 


^ c' = a 


'±b' 












ti 2 


<->2 





17. Eliminating a 
feedback loop 



-? — Q-M. — K& \-r*~ 

b\ I 

c = a±b 



KfGo 



a * 


tfiGj 


d ^ 




1T(X 1 G 1 KK 2 G 2 ) 





18. Special form 
of 17 



Q- 5 -*— *i G 



&A c = a±b 



ifiG! 


d ) 





19. Special form 
of 17 



6 



-N a±6. 




d^ 


1 








Kl G l 











a -» 


1 


d 




l + if 1 G 1 





20. Inserting a 
feedback loop 
to replace an 
element 



* *iCi 



a-6 


X1G1 

1-KjGa 




d 


? 




* 






6 = d 



21. Different form 
of 20 



XjGi 



-sa- 


6 5 




d 3 










-J 1 
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internal loop, thus replacing a loop by a closed loop transfer function, 
which has no takeoff or feedback points. 

Examples to Illustrate Transformation Rules. 

Example 1. The forward elements Gi, G 2 , and G s , in Fig. 25a may be 
combined by multiplication as shown by eq. (56) and Fig. 256: 



(56) 



C M x M 2 C 

— = — • — • — = G1G2G3 = G. 

E E M x M 2 



■zi&z-*- 



B 



M^ 



G 2 



M 9 



H 



(a) 



G 3 



B 


^ E > 


G 
H 




C 


J > 















R 



l + GH 



(0 



(b) 

Fig. 25. (a) Simple closed loop system; (6) combinations of forward transfer functions; 

(c) system in simplest form. 

In practice, loading of one element by another must be considered. 
Figure 256 is further reduced to Fig. 25c by use of eq. (57) : 



(57) 



C 



G 



R l + GH 



Direct or unity feedback is also common and represents a particular case 
of eq. (57), where H = 1. 

Example 2. Reduction of a complex diagram is shoAvn in Fig. 26. 

Note in the first reduction, Fig. 266 that the block diagram is altered to 

include an additional G4 element for mathematical simplicity, although 

the signal flow and algebra is identical. Also note in Fig. 26d that a 

positive feedback is accomplished by using eq. (58), except that the H 

has a negative sign. 

G2G5 

(58) G e = — 

1 — G2G5H1 
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H 2 










R tr^ E - 


G X 


Ml lc 


M 2 


G 2 


B 2 

M 3 -X M 4 


G3 






Ga 




C 


-V 

M 5 


J ' 


Bi 




4> 
Hi 


M 5 


> 


f 





















(a) Original system 







Mi 




B 2 


-H 2 


a 


G, 


l 


^5 


M b 


G 4 




R +~E 


Gi 


-io^ 


G 2 


M 3 -±M 4 


G 3 




C 




r 


4- , 


J 


4- 


■ 


' 
















vr > 




^5 


■Dl 




Hi 


M 5 









































(b) First reduction 



*ic£ 



— A 

M, 



Mi 



+ A 



M, 



- G3 

5- I + G4//2 



G R = 



M fi 



#i 



M* 



(c) Second reduction 



fl + 



?? 



M, 



> Gi 



M, 



n — ^2^5 
^6" I-G2G5H1 



M s 



G 4 



(d) Third reduction 



R 


+ 


GiGe 


M 5 > 


G 4 


C 






"' l + GiG 6 








(e) Final 
Fig. 26. 


reduction 
(Ref. 3a). 
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Example 3. For systems with multiple input and/or disturbances, the 
superposition theorem is used. The example of Fig. 27 is used to show the 
response C as a function of two inputs. 



-K>— — * 



+ * 
Mi 



G 2 



Gi 



Ri 



■*fr 



- B 



H 



(a) System with multiple inputs 



B 



R 2 + 



*? 



(b) With # 2 = 
M 2 



M, 



G, 



Gi 



H 



LetR 2 = as in Fig. 276: 



(59) 



Let Ri = as in Fig. 27c: 



(60) 
Combining inputs: 

(61) 

4. SYSTEM TYPES 



C 
C 



(c) With Ri = 
Fig. 27. 

GiG 2 

TVg^h' 



Go 



R 2 1 + G\G 2 H 
GiG 2 R\ -\- G 2 R 2 



C = 



1 + G^ff 



"i £ > 


Gi 


Mi ^ 


G 2 




C 


J * 






















H 



















Definition of System Types 

The idea of the functional similarity of seemingly different transfer 
functions is strengthened by classification into types. Three common types 
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are ones in which the following conditions are obtained after the transient has 
subsided: 

Type 0. A constant value of the controlled variable requires a constant 
actuating error signal. 

Type 1. A constant rate of change of the controlled variable requires a 
constant actuating error signal. 

Type 2. A constant acceleration of the controlled variable requires a 
constant actuating error signal. 

These characteristics may be identified in terms of the transfer function. 
For a simple closed loop system with direct feedback the error signal 

(62) E(s) = R(s) - C(s), 

where R, the reference input signal, is compared with C, the output signal. 
The forward transfer function 

C(s) 

(63) 0(8) = 



is of the general form 
(64) G(s) = 



K(l + ais + a 2 + 



■) 



s n (l + b lS + b 2 s 2 + b 3 s*+.--) 



The value of the integer n in eq. (64) is equal numerically to the type of the 
system. 

Complex plane plots may be obtained by replacing s by ju in eq. (64). 
The nature of the plots as u — -> will be representative of the type of 
servomechanism studied, as illustrated in the following section sub- 
division, Typical Complex Plane Plots. 

Typical Complex Plane Plots 

A type servomechanism representative plot is given in Fig. 28 (Ref. 34). 
At o) = 0, the transfer function G(ju) is on the positive real axis and has a 



Imaginary 




co = 



w increasing 



Fig 28. Representative complex plane plot for type servomechanism system (Ref. 34). 
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finite value K p . Generally as co — > <», G(ju>) traverses the fourth and 
then the third quadrants and approaches the origin. 

A type 1 servomechanism representative plot is given in Fig. 29 (Ref. 34). 
For this plot as co — » 0, the polar plot of G(ju>) approaches minus infinity 
on the imaginary axis. Generally as co increases toward +<», (r(jco) enters 



Imaginary 



-1+jO 



G(ju) 




CO = +00 



Re 



G(j<a) = 

ioo(i +y§3> 



jua+wa + ;£)o + ;£> 



20' 



50 



co increasing \) 



Fig. 29. Representative complex plane plot for type 1 servomechanism system (Ref. 3). 

the third and then the second quadrant. The type 1 servomechanism 
when used for a position control system may also be called a "zero dis- 
placement-error system" meaning that the output has the desired value 
of displacement, in contrast to the type servomechanism, where an 
error proportional to the desired amount of displacement is necessitated. 

A type 2 servomechanism representative plot is presented in Fig. 30 (Ref. 
34). For this plot as co —> 0, the plot of G(jw) approaches minus infinity 



Imaginary 




0a>) 2 (i+y^) 
Fig. 30. Representative complex plane plot for type 2 servomechanism system (Ref. 3). 
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on the real axis. The plot may be closed from w = 0+ to a> = 0— by a 
circle of infinite radius traversed in a counterclockwise direction, as in- 
dicated by the dotted line. The type 2 servomechanism has a "zero- 
velocity error" characteristic since it is able to maintain a constant output 
speed with no actuating error. It is also capable, like the type 1 servo- 
mechanism, of maintaining a constant output position without actuating 
error. 

Typical Application 

Examples of type servomechanisms are speed regulators for d-c motors 
and jet engines and other forms of regulators controlling voltage, current, 
or temperature, where proportional controllers are employed. 

Examples of type 1 servomechanisms are position control systems with 
such integral controllers as d-c motors, hydraulic motors, and hydraulic 
valve-piston linkages. Other examples of type 1 servomechanisms are 
speed control systems such as for a jet engine with proportional and in- 
tegral control. 

Examples of type 2 servomechanisms are position control systems in which 
a pilot motor is employed to drive a control element, whose position 
controls the speed of the main drive motor that supplies power to the 
load being positioned and torque motors with series compensation. 

Block diagrams of each of these servomechanism types are given in 
Ref. 34. The following paragraph is in substance from Servomechanism 
Analysis by G. J. Thaler and R. G. Brown (Ref. 35). 



Table 18. Characteristics and Applications of Types 0, 1 and 2 
Servomechanisms (Ref. 35) 



Type 


Locus 


Error 




System Characteristic 


Characteristic 


Application 





Closed. 


Position error at all 


Static positioning sys- 






times. 


tems where high ac- 
curacy is not impor- 
tant. Some regula- 
tor systems. 


1 


Open. The low-fre- 


No static error. Lag 


High-accuracy static 




quency end of the lo- 


error when operated 


and dynamic posi- 




cus goes to infinity 


at constant velocity. 


tioning systems. 
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In general, the complexity of equipment, cost, and difficulty in design 
increase greatly with the more advanced type of systems. Type 1 servo- 
mechanisms are therefore more common than any of the others. Oc- 
casionally, accuracy requirements will justify the type 2 system, and in 
other cases where high accuracy is not essential a type system is more 
economical. Table 18 summarizes characteristics and applications of 
types 0, 1, and 2 servomechanisms. 

5. ERROR COEFFICIENTS 

One of the important figures of merit of a system is its accuracy under 
various conditions. By accuracy is meant the ability of the system to 
minimize the error between the actual output and the desired output. 
The usual types of accuracy specified for a control system are its static 
accuracy and its dynamic accuracy. Static accuracy is the accuracy for 
the output or one of its specified derivatives in a steady-state condition. 
Dynamic accuracy is the accuracy existing during transient conditions of 
the output and of its derivatives. 

Static Error Coefficients 

A static error coefficient may be defined as the ratio of the steady-state 
constant value of the output or of one of its constant derivatives to a 
constant applied error. The static error coefficients are then: 

Position error coefficient, 

Output, c . C(s) 

K p = = lim 

Applied error, e s->o E(s) 

for constant output, c. 



Velocity error coefficient, 



Velocity of output, c sC(s) 

K v = = lim 

Applied error, e s->o E(s) 



a dc 
for constant velocity of output, c = — ■ 

dt 

Acceleration error coefficient, 



Acceleration of output, c , s C(s) 

K a = = lim 

Applied error, e s->o E(s) 



for constant acceleration of output, c = —^ 



A 
dt 2 
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K p , K v , and K a are respectively the gain constants of type 0, 1, and 2 control 
systems. 

For a sinusoidal applied error e, the error coefficients for types 1 and 2 
control systems may be defined in terms of the maximum velocity and ac- 
celeration of the output c as follows : 

Velocity error coefficient (type 1): 

(65) K' v =]Jm- Caux 



«->o e c v 



where e \& = e at time of max c. 



Acceleration error coefficient (type 2) 
(66) K' a = lim 



"max 



a>->o e I V 

I c max 

where e I v = e at time of max c. 

I ''max 

In the limit K' v and K' a are identical in value to the values of K v and 
K a obtained for constant c and c. 

Table 19 presents a comparison of errors for various types of controlled 
motion in which c, c, or c is constant or c is oscillating sinusoidally at a 
much lower frequency than that corresponding to the shortest time con- 
stant of the control system. 

Table 19. Comparison of Steady-State Errors (Ref. 34) 

Type of control system 

Limit G(s), s — >• 

Error for constant output, c 

Error for constant velocity of output, c 

Error for constant acceleration of output, c 

Maximum e for sinusoidally varying c = 

Cmax sin uit C max /Kp ioC maK /K v a) C ma , K /K a 

Dynamic Error Coefficients 

A form of dynamic error coefficient is defined as the ratio of the input 
of one of its specified derivatives to the component of the error which may 
be assigned to it during a dynamic condition. That is, the error may be 
expanded in a series in terms of the input and the derivatives of the input. 
The dynamic error coefficients are then the reciprocals of the coefficients 
of the various derivatives since they indicate proportionality between 
dynamic error components and input derivatives. 
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Writing the transform of a system with unity feedback gives 

E(s) 1 



(67) 



R(s) 1 + G(s) 
E(s) may be expanded from the ratio of two functions of s, 

R(s) 



(68) E(s) 



1 + G(«) 



into R(s) operated upon by the Maclaurin series expansion of 1/[1 + G(s)] ) 
or by simply dividing the numerator by the denominator, 

(69) E(s) = — i— - R(s) + -^ sR{s) + i- s 2 R(s) • . • , 

1 + iv p Ai A2 

which is valid near s = (the steady state). Let K = 1 + -Kp- 

if , -K\, -K^2) * • • are commonly called dynamic error coefficients (Refs. 
36 and 37) of the system. Note. Strictly speaking, Kq, K\, K 2 , etc., are 
not "dynamic" error coefficients since the transient terms were lost upon 
the series expansion of eq. (68). More correctly, they might be termed 
steady-state error coefficients. The term dynamic is common usage. 

Kq, K\, K 2 may contain not only the values of the static error coef- 
ficients K p , K v , and K a , respectively, but also expressions involving the 
static error coefficients and the time constants of the system. Therefore, 
high gain alone is not sufficient for accurate dynamic performance, for 
low system time constants are also important for this purpose. 

Example. Dynamic error coefficients. For the position servo of Fig. 11 : 

K v 

(70) G(s) = 



s(l + T lS ) 
where K v = K/D, 
T x = J/D. 



From eqs. (68) and (70) for unity feedback 

R(s) s + T x s 2 

1 + G(s) ~ K v + s + T x s 

K v K v 2 _ 



(7i) m = ^ ;_ = — — ~^~ 2 r( S ), 



(72) E(s) = -U#( s ) + 

1Y V 



s 2 R(s) + 



The dynamic error coefficients in this case are K v and K V 2 /(K V T 1 — 1), 
showing that here the system time constant produces an acceleration 
component of error proportional to the time constant. 



Locus 
Identifi- 
cation 

6-12 



6-12-18 



6-12-6-12 



6-12-18-12 



12-6-12 



Table 20 Servo Error Coefficients (Ref. 39) 

[ e(t) = — r(t) + — — r(t) +r—--^ nt) + — —„ r(t) + • • • (K = 1 + K p and K p = =o for all servos in table) ] 
L Ko K\ at a 2 at 1 a 3 at 6 J 



Transfer 
Function G(s) 



s(s + o> 2 ) 



0)10)2(0)3/0)4) (s + 0)4) 
s(s + W 2 )(S + 0)3) 

coi(s + 0)2) 
S 2 (S + 0)3) 



1_ 

1 

coi 



s(s + oi 2 )(s + C0 3 ) 

o>ioi 2 (s + 013) 0)4 

S(S + W2)(S + 0)4) 0)10)3 



1 



1 

COI — w 2 
wi 2 co 2 

C01C0 2 + 0>10>3 — C03C0 2 



1 

o> 2 — 2coi 



o>i — 2coia' 2 — 2wiW3 + co 2 a>3 



0)10)3(0)2 4" 0)4) — 0)20)4(0)1 + 0)4) 0)10)3(0)10)3 — 2a>4 — 2o)20)4 — 0110)4 — 0)10)2) -f- 0)20)4(0)1 -f- 0)4) 



0)10)4(012 + 0)3) — o) 2 o)3(wi -f- 0)4) 0)10)4(0)10)4 — 2o>30>4 — 2o) 2 o>4 — o)io>3 — o)iO) 2 ) -+- 0)20)3(0)1 + 0)4) 
o>i 0)2013014 o>i 014 012 



013 
0)iO)2 
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Error Calculation for a Given Input. The error may be calculated 
for a given input when r(t), r'(t), r"(t), etc., are known or calculated. First 
the transforms R(s), sR(s), s 2 R(s), etc., are evaluated from the input and 
its derivatives by the formula for real differentiation: 

n 

(73) s n F(s) = £[f M (t)] + E/ (fc_1) (0 +)s n - fc -> JB[f (n) (0]. 

Then the values of s n R(s) are substituted into eq. (69) to obtain the 
Laplace transform of the error, E(s). The inverse Laplace transform taken 
term by term then yields the error as a function of time, e(t). If impulses 
at t = are neglected, 

(74) e(t) = -i- r(t) + i- r'(t) + -1 r"{t) + • • • . 

A -ft-! i£ 2 

When an analytical expression is not available, the values of r{t), r'(t), 
r"(t), etc., can be estimated from a graph of the input function. 

Table 20 gives the literal values of dynamic error coefficients of eq. 

(74) for some common systems with unity feedback. The locus identifica- 
tion in the table means the succession of decibel per octave slopes with 
increasing co of the straight-line approximation to the log magnitude versus 
log frequency diagram for G(jw). 

An example of the evaluation of the dynamic error is given in Table 21 
(Ref. 40). Use of eq. (74) is illustrated further in Refs. 36 and 38 where 
six examples are given. See also Chap. 23. 

Method of Calculating Accuracy of Dynamic Error Coefficients. 
As previously stated, the error coefficients of eq. (69) or (74) are evaluated 
for s = and hence eq. (74) describes the steady-state error. Further- 
more, the usefulness or validity of eq. (74) depends upon the nature of 
r(t) and also upon the rapidity of convergence of the series. The remainder 
of this section is in substance according to Arthurs and Martin (Ref. 
41). The correction term R n +i involved in expressing the error e(t) 
by a finite number, n + 1, of coefficients (or terms) of eq. (74) (where 
r, r', r", • • • , r n are continuous) is 

(75) R n+1 = f r n+ \r)g n+1 (t - r) dr, 

J—<x> 

where by recurrence gi+\(t) is obtained as [— u{t) f gi(y) dy], u(t) being a 

unit step and g (t) being the response, such as e(t), of the system to a 
unit impulse. 



FUNDAMENTALS OF SYSTEM ANALYSIS 



20-75 



Table 21. Dynamic Error, eit), for Servo op Fig. 11 When Tracking Airplane 
at Same Elevation (Ref . 40) 



j_^; 




Input Angle r(t) and Two Derivatives 



r(0 



dt 2 



[KOI 



d 2 



Ht)] 



tan" 



At 



A/L 



dt 2 
-2(A*/L s )t 



L 1 + (At/L) 2 [1 + (At/L) 2 ) 2 

Values of Error Coefficients K 0) K\, K 2 (see eqs. 69 and 72) 

J_ 1 J_ _1_ 

K 2 



K 



1 + K- 




K v 



K v 



1 



Dynamic Error, e(t) (see eq. 74) 
4/L 



m = T-ii r A/L i - r^i _ jli r 2(AVL 3 x -i 

eW LtfJ LI + U*/L) 2 J IK V K 2 \ L[l + (^/L) 2 ] 2 J " 1 " " 

An upper bound for the correction term .Rn+i may be found by replacing 
r n+1 (j) by |r n+1 (r) | max and performing the integration of eq. (75) (a valid 
procedure for many functions g n +i(t — r)). 

When r, r', r", • • • , r n suffer at most step discontinuities, the response may 
be expressed by the first n + 1 terms of eq. (74) plus R n +i of eq. (75) plus 
the expression: 

n M i 
i=0 k=l 

where Mi is the number of step discontinuities of r l (t), A,fc is the magnitude 

of the discontinuities, and Uk is the time of the kth. discontinuity. The 

contribution of impulses can be added in separately. 

The response at any time t may be written in three parts : 

1. A finite number of terms of the equivalent of the familiar dynamic 

error expansion, eq. (74), 
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2. A corresponding finite set of transient terms which accounts for 
possible discontinuities in the arbitrary forcing function and its deriva- 
tives. 

3. A convolution integral which clearly places in evidence the exact 
inaccuracy in the response involved in using a finite number of coeffi- 
cients (equivalent to the familiar error coefficients). 

The above expression results from a closed expansion of the convolution 
integral to which the response at any time t may be equated. The useful- 
ness of this expansion of the response into three parts lies in the fact that 
in many problems R n +i contributes a small portion of the total response. 
Consequently R n +i may either be neglected or crudely approximated 
without introducing appreciable inaccuracy in the total response. In such 
cases the process of convolution to obtain the response is replaced by 
differentiation and summation. 

If r n+1 (r) in eq. (75) is replaced by its maximum absolute value, then 
for many functions g n +i(t — t) it may be shown that integration yields 
an upper bound, 

|^n+l|max =|^n+2(0)| | r n+1 (r) | max = | C n+1 | | r n+l {r) | maX , 

where c n+1 is the coefficient of r n+1 (t) in the expansion of the response and 
of s n+1 in the Maclaurin series for the system function such as E(s)/R(s). 
Example. Dynamic Error Expressed by Expansion of the Convolution 
Integral. Illustrate the expansion of the dynamic error for a case in which 
there exists a known solution. Let E(s)/R(s) of eq. (67) be B/(s + b), 
the response characteristics gi+i(t) being 

B (~zi) e ~ btu{t) > 

and the Maclaurin series expansion being 



5 b \-b) 



(see eq. 69). Then the dynamic error may be expressed by n + 1 terms 
of the error expansion 



h b \-bJ 



r*(t) 



(see eq. 74) plus the transient terms 

n Mi / 1 \ l + 1 



n M t / I \ l 

i=0 k=\ \ — 0/ 



,-Ht-t ik ) 
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plus the remainder 

/ \T n+ Hr)][B(^j n e- b ^\i{t-r)]dr 

(see eq. 75), which may be bounded by 

B / 1 \ n+1 

in which is recognized the (n + l)th coefficient of the Maclaurin series for 
E(s)/R(s). 

Let R (s) be A/(s + a). Then the input and its derivatives r l (t) are 
given by (—a) l Ae~ at u(t), and for each value of i, r l (t) has one discontinuity 
(Mi = 1) at time tn = of magnitude An given by (—a) 1 A. Also 
k n+1 (T)|ma X isgivenbya rl+1 ^. 

The dynamic error then may be written as 



I/O© 



-J (e~ at ~ e~ bt ), 

with the remainder R n+ \ bounded by A(B/b)(a/b) n+1 . This series for 
the error converges rapidly and is useful for a <<C b, corresponding to 
an input which is slow compared with the system response characteristics. 
As n — » oo, the dynamic error expansion — » AB[(e~ at — e~ bt )/(b — a)], 
which checks with the inverse Laplace transform of AB/(s -\- a)(s -\- b). 

Another example of the expansion of the convolution integral to yield a 
time response in three parts is given in Ref. 41. 

Relative Usefulness of Error Coefficients. Both static and dynamic 
error coefficients are treated in Ref. 34. Dynamic error coefficients are 
also treated in Refs. 37 and 36, which includes a treatment of the ap- 
proximate relations, for small overshoot, between the time delay and rise 
time of the step function response and the dynamic error coefficients. 

Dynamic error coefficients Ko, Ki, K 2 • • • have been defined by eq. 
(69) and have the same values for all error constants up to and including 
the one which is nonzero and finite in the classical definition of static 
error coefficients, K p , K v , K a , " • (as given in Static Error Coefficients). 
This definition of dynamic error coefficients by eq. (69) has the advantage 
of giving additional information about a system, because the value of 
any constant is not forced to be zero whenever the preceding constant is 
nonzero and finite as is the case with static error coefficients K p , K v , K a , 
• • •. (These coefficients K , Ki, K 2 • • • are the same in the steady state 
as the dynamic error coefficients defined for any time t at the beginning of 
this section subdivision.) 
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Relationship between Dynamic Error Coefficients and Roots of 
System Equations. This section is, in substance, from Automatic Feed- 
back Control System Synthesis by J. G. Traxal (Ref. 42). 

As a result of the relation between C/R and E/R, there results the 
Maclaurin expansion 

C(s) 11 1 „ 

(76) -1-1=1 s s 2 . 

R(s) K K x K 2 

The relation between the dynamic error coefficients K , K\, K 2 and the 
poles and zeros of the closed loop expression C/R is readily determined 
if C/R is written in factored form. 

,„, C(s) (s + 2i)(s + z 2 ) • • • (s + z m ) 

y* 1 ) T77T = K 



R(S) (s + Pi)(s + p 2 ) ••• (S + Pn) 



where the zeros lie at —Z{, the poles at — p t -. 

The solutions for the dynamic error coefficients are: 



(78) K = ^- 



n Pi - k n zi 



where JJ indicates the product of all factors from i = 1 to and including 



i=i 



-?, 


1 JL 1 


m X 


%2~ 


= — + H — 





i = m and for cases where K p — > °o (K = 1 + i^ p ), 

-i n I m -i 

(79) -=E--Z- 



(80) 



Equations (79) and (80) are of basic importance in servo synthesis for 
they represent the correlation between the dynamic error coefficients and 
the system response characteristics, specifically the time delay and rise 
time of the response of the system to a step function. In addition the 
two equations indicate the manner in which lead and integral equalization 
permit control over K\ and K 2 without affecting relative stability. Gener- 
alized (dynamic) error coefficients are treated in greater detail in Ref. 36, 
and their relation to closed loop roots is treated at length in Ref. 42. 
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Guillemin's Method. Equations (79) and (80) are of basic importance 
in feedback control system synthesis by Guillemin's method, which is 
described in Chap. 23. 

In the first step of this method the closed loop transfer function is de- 
termined from the specifications for frequency response and transient 
response. In Chap. 23 use is made of techniques for obtaining with the 
aid of these two equations the zeros and poles for compensation required 
to obtain the desired closed loop transfer function. 

6. ANALYSIS OF A-C SERVOS: CARRIER SYSTEMS 

In a d-c servo the signals are directly proportional to the instantaneous 
amplitude whereas in an a-c system the signals are modulated carrier 
waves, and the information is carried in the modulation. For instance in 
an amplitude modulated system the envelope of the modulated wave 
contains the signal information. Owing to the convenience and simplicity 
of using a synchro or chopper circuit as a modulator and a two-phase 
motor as a demodulator, many carrier systems use suppressed-carrier, 
amplitude modulation. Frequency and phase modulation are not yet in 
common use although they offer theoretical advantage for null detection. 

The value of an a-c system lies in the possible use of sensitive, accurate, 
low-force level sensors, inexpensive and relatively easily produced as am- 
plifiers, and power elements with low maintenance requirements. 

Basic Types of Elements. Three types of elements are encountered in 
carrier systems : 

Type 1. Elements in which both input and output are modulated 
carriers. 

Type 2. Modulators which have inputs at signal frequencies and 
outputs which are modulated carriers. 

Type 3. Demodulators which have modulated carriers as inputs and 
have signal frequencies as outputs. 

Figure 31 shows several electronic circuits that can be used as modulators 
and demodulators. In addition the mechanically tuned or electrome- 
chanical choppers are used extensively. The a-c servo motor also serves 
as a demodulator and the a-c tachometer acts as a modulator. 

Two-phase servomotors and a-c tachometers are analyzed in Refs. 
45-47 and 49-51. An accurate mathematical representation of these 
elements is complex and simplifying assumptions and analogies are com- 
monly used. 

Simplifying Assumptions. The majority of work done with a-c feed- 
back systems has been on suppressed carrier systems. For this type of 
system the following assumptions are normally made: 
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d-c signal 
Input (modulator) 
Output (demod.) 



a-c 
ref. 




a-c signal 



Vacuum Tube Diode 



? a-c ref. 

1 <SL<msLr 




d-c signal 



Diodes 



I 



a-c signal 



I 






Square k 
wave ref, 




d-c signal 



Transistor 
Fig. 31. Modulators and demodulators. 
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Input 

a-c (demod.) 

d-c (mod.) 






BIAS 



BIAS 




Vacuum Tube Triodes 




Output 

d-c (demod.) 

a-c (mod.) 



Input a-c/d-c 




Input 



X 



a-c ref. 



Magnetic-Amplifier 



q o— . Output 
d-c/a-c 



T 



BIAS 



BIAS 



1 



X 



Output 



Transistor 
Fig. 31. Modulators and demodulators (continued). 
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1. Modulators generate perfect suppressed carrier signals; i.e., har- 
monics besides the sideband frequencies, self-generated noise, and serious 
phase shifts in the modulator are neglected. 

2. The motor acts as a perfect demodulator with a prescribed static and 
dynamic relationship between the driving voltage envelope and the out- 
put shaft position. 

3. The carrier frequency and magnitude remain constant. 
Suppressed Carrier System. A simple suppressed carrier open loop 

system is shown in Fig. 32. If the input to the preamplifier is the product 
of the carrier and the reference input, then 

(M cos au c t)(V cos w c t) = M F(cos aoj c i)(cos oj c <). 

Expanding this yields 

(81) MF[cos (1 + a)<a c t + cos (1 - o)w c *]. 

As indicated by eq. (81) this type of modulator has an output containing 
only the two sideband frequencies, the sum and difference of the modulating 





1 Reference 
1 Vcos 03 c t 




Reference l 
Vsinwc* 1 




Input _ 


Bridge 
modulator 




A-c pre- 
ampl. & stab. 




Ampl. & 
two-phase 
servo motor 


Output 


M cos au c t 


r 




K cos (aa) c t + <t>) 



V MV 
N -y [COS (1 + O) W c t + COS (1 - a) <t) c t] 

Fig. 32. Open loop carrier system (Ref. 3b). 

and carrier frequencies, (1 — a)co c and (1 + a)co c , and no carrier frequency, 
co c . This gives rise to the name suppressed carrier. 

A typical suppressed-carrier feedback system using tachometer stabiliza- 
tion is shown in Fig. 33. The signal equations are indicated at the sig- 
nificant points in the system. 



M cos ao) c t 



> Synchro. 



K cos (au c t + <W 




K 2 cos (aa} c t + 3 ) cos to c t 

K^ cos (ao) c t + 02) cos w c' 



Ampl. 



O 



Ampl. 



Two- 
phase 
motor 



K cos (aw c t + <t>) 



—au c t KiS'm (ao) e t + 0jj cos u c t 



A-c 
tachometer 



Fig. 33. Typical a-c servo with tachometer feedback (Ref. 36). 
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System Analysis and Design. For the purposes of system analysis 
the a-c components can usually be treated in the same manner as analogous 
d-c components. For instance as shown in Fig. 34, the speed-torque 
curves of a d-c and an a-c machine are analogous, and the analysis of the 
a-c machine can proceed in the same manner as the analysis of the d-c 
machine. See Table 5, Sect. 1. Note, however, that the a-c machine 





Fixed 
source 



' Synchronous speed 

Applied voltage 
>^~ Rated voltage 




Torque 

Torque-speed characteristics 

of d-c shunt motor 

Fig. 34. 



Torque 

A-c two-phase motor linear 
approximation to d-c motor 

D-c analogy to a-c two-phase motor. 



characteristics are nonlinear and that to derive a linear transfer function, 
the analysis must be carried out on a linearized, incremental change basis. 
The linearization methods of Sect. 2, Chap. 25, can be used. 

There are torques at other frequencies besides the signal frequency 
produced in the motor. These torques are at frequencies (2 -f- a)o3 c and 
(2 — a)o> c and normally because of the high frequency and low amplitude 
produce little mechanical motion. However, the associated currents can 
produce important heating effects. Similarly because the motor tends to 
discriminate against quadrature control signals, the currents produce little 
torque but the heating and/or saturating effects of quadrature currents 
can be important. 

The a-c and d-c analogy can be extended to a-c tachometers. An a-c 
tachometer with a control signal of a frequency a« c and an amplitude pro- 
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portional to aco c affects system performance in a manner similar to a d-c 
tachometer. 

Alternating-current stabilizing networks and a-c system stability are 
treated in Chap. 23, Sect. 3. 

Noise, quadrature voltage or carrier phase shift, variations in carrier 
frequency, and pickup present major problems in a-c system design, and 
their consideration dictates as much as the stability analysis the form and 
characteristics selected. As a result it is desirable to define the environ- 
ment arid operating requirements before investigating the system stability. 
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1. INTRODUCTION 

Definition of Stability. A stable system is one wherein all transients 
decay to zero in the steady state. An unstable system is here loosely 
denned as one in which the response variable increases without bound 
with a bounded signal input. 

Reason for Stability Analysis. The primary objective of a control 
system design is to devise a system such that a controlled variable is 
related to a command signal in a desired manner within permissible toler- 
ances. If power elements with reliable, unchanging characteristics were 
available, the problems of control system design would be much simplified. 
Since, in the main, the characteristics of power elements change with 
time, temperature, load, pressure, etc., a feedback element is employed to 
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remove the deleterious effects of change in element characteristics. To 
improve performance of the system, a -natural solution is to increase the 
gain or amplification in the system. The combination of a closed loop and 
high gain leads to problems of instability. 

Purpose of Stability Analysis. To be a satisfactory control system, 
the system deviation resulting from any normally encountered deviation 
stimulus must reduce with increasing time to a small value within accept- 
able tolerance. It is the purpose of stability studies to indicate a system's 
dynamic behavior, and if this behavior is improper or inadequate, the 
studies should point the way toward proper system revision to improve 
performance. 

Methods of Stability Analysis. The methods of studying stability 
presented in this chapter are restricted to linear systems. A linear system 
is one in which the output due to simultaneous inputs is the same as the 

sum of the several outputs due to 

Q-gfs) 1 Gfe) | f C ^ s) the inputs acting alone. In other 

words, a linear system is one which 
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G(s) 




C(s) 
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H(s) 













may be described by ordinary linear 
differential equations, wherein the 

Fig. 1. General negative feedback system. theorem of superposition holds true. 

For nonlinear systems, see Chap. 25. 
Consider the general feedback system shown in Fig. 1 ; s is the Laplace 

transform complex variable. The transfer characteristics are given by 

C(s) G(s) 

(1). 



B(s) 1 + G(s)H(s) 

The stability of a system is uniquely defined by those values of s which 
make 

(2) 1 + G(s)H(s) = 0. 

All the methods of stability analysis, therefore, confine themselves to in- 
vestigation of eq. (2), in one fashion or another. The techniques can be 
classified in two general categories : those which obtain the explicit values of 
the roots of eq. (2) and those which obtain information about the bounded 
regions wherein all the roots lie. In the first category belong the classical 
approach and the root locus method. In the second category belong Routh's 
criteria, Nyquist's criteria, Bode's method, and many others. The relative 
merits of each will be discussed as each method is examined in detail. 

2. CLASSICAL SOLUTION APPROACH 

As shown in Chap. 20, it is possible to relate the controlled variable to 
the command (reference) variable by a differential equation. For the sta- 
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bility studies involved here, assume this is a linear differential equation of 
the form 

d n x(t) d n ~ l x{t) 

(3) a °~^ _ + ai ~^ r " + " ,+ an:C(0 

d m y{t) d m - 1 y(t) 

Since this equation is linear, the solution may be broken into the sum of 
two solutions, the particular solution and the complementary or homogeneous 
solution. 

The particular solution, x ss , also called the forced response or the steady- 
state solution, is of the form 

(4) x ss = f(x). 

In other words, the forced response is of the same character as the refer- 
ence. For example, if y is sinusoidal, x ss is also sinusoidal. 

Characteristic Equation. When the operator p is substituted for d/dt 
and y is set equal to zero, eq. (3) becomes 

(5) a p n x(t) + aip n_1 a;(0 + a 2 p n ~ 2 x{t) -\ \- a n _ lV x{t) + a n x{t) = 

or 

(6) [a p n + a^- 1 + a 2 p n - 2 + • • • + a n _ x p + a n ]x(t) = 0. 

This is the characteristic equation (see Chap. 20) of the system. The 
complementary solution, x t , also called the homogeneous solution or transient 
solution, is of the form 

(7) x t = A ie p S + A 2 e p t + A 3 e p > 1 + • • • + A n e p nK 

The exponents p lt p 2 , P3, • • •, p n are the roots of eq. (6). The coef- 
ficients Ax, A 2 , A s , • • • , A n depend upon the initial conditions of the sys- 
tem and the forcing function, y. 

Note that when multiple roots occur, say p\ = p 2 , the transient solution 
is of the form 

(8) x t = A x e p S + A 2 te p J + A 3 e p ^ + • • • + A„eV. 

9 

The total solution is the sum of the two parts 

(9) x = x ss + x t . 

Relation of Stability to Characteristic Equation. Instability has 
been defined as the output becoming large without bound for bounded 
input. Since the steady-state solution is of the same character as the 
forcing function, only the transient solution can provide terms which in- 
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crease without bound for bounded input. This occurs when any of the 
roots p 1} p 2 , P3, • - -, p n have positive real parts because the corresponding 
exponential terms in eq. (7) or (8) tend to infinity as t becomes infinite. 
Because A\, A 2) A 3 , • • •, A n are finite values depending on initial condi- 
tions and y ss is bounded for a bounded input, system stability is dependent 
only upon the nature of the characteristic equation of the system! In other 
words, system stability is uniquely determined by the behavior of the exponen- 
tial terms in the transient response given by eq. (7) or (8). 

a. If all the roots have negative real parts, all the exponential terms 
decay to zero as time increases. This is a stable system. 

b. If any of the roots have positive real parts, the corresponding ex- 
ponential terms increase without limit. This is an unstable system. 

c. If any of the roots are purely imaginary, the corresponding terms 
oscillate at constant amplitude. This condition is the dividing point be- 
tween a stable and an unstable system. It is here also considered un- 
stable. 

d. If it so happens that multiple roots occur, i.e., p\ = p 2 , which are 
purely imaginary, the output increases without bound. Again this is an 
unstable condition. 

The fundamental problem in ascertaining system stability is therefore one 
of determining the nature of the roots of the characteristic equation of a given 
system. The straightforward method of determining stability of a system 
consists of the following steps : 

a. Write the differential equation of the system relating input and out- 
put variables. 

b. Substitute p for d/dt and equate the input signal to zero. This is 
the characteristic equation of the system in operational form. 

c. Obtain the roots of the characteristic equation with assigned values 
for all constants. 

d. Examine the roots. If all roots have negative real parts, the system 
is stable. If any of the roots have zero or positive real parts, the system is 
unstable. 

Figure 2 shows the regions of root location for stable and unstable 
systems. 

Note. If the Laplace transform method of analysis had been used, the 
conclusions would have been identical except that the complex variable 
s would replace the operator p. Equation (2) would yield: 

(10) [a s n + a^- 1 + • • • + a n ]X(s) = [b s m + & lS w_1 + • • • + b m ]Y(s). 

As an input-output ratio similar to eq. (1) this is 

C(s) X(s) b s m + b 1 s m - 1 +..-+b m 



R(s) Y(s) a s n + a^ -1 + a 2 s n ~ 2 H \- a 
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Solution of the equation resulting from setting the denominator of eq. 
(11) to zero is exactly the same as solution of eq. (2). [1 + G(s)H(s)] 
is a fraction of polynomials in s where the numerator polynomial is the 
characteristic function of the system. As shown in Fig. 2, stability is 
uniquely defined by those values of s which satisfy eq. (2). Because of 
more universal acceptance, the complex variable s will be used in place of 
the operator p in all the following methods of stability analysis. 




Fig. 2. Pictorial representation of stability definition. 

Relative Merits of the Classical Solution Approach. This method 
of determining stability has the advantage of being theoretically exact, 
but suffers from two major disadvantages: 

a. A great amount of labor is required to factor equations of degree 
higher than 3. 

b. To factor any higher order equations, the coefficients must be nu- 
merical values. The loss of system parameters in literal form obscures the 
ways to improve system performance should redesign become necessary. 

3. ROUTH'S CRITERION (Refs. 1, 2, 3) 

In 1877 E. J. Routh developed an algebraic method for determining 
whether a polynomial has roots with positive real parts. This method 
does not reveal the exact values of the roots but shows the bounded regions 
wherein they are located. Reference to Fig. 2 shows that this is all that 
is necessary to determine whether a system is stable or not. If all roots 
lie in the left half s-plane, the system is stable. 

Application of the Routh Criterion. 

Step 1. Write the characteristic equation in the form 

(12) [a s n + ais"" 1 + a 2 s n ~ 2 + • • • + o„_is + a n ]X(s) = 0. 

Remove all the zero roots, i.e., the roots that occur at s = 0. If the 
zero roots do occur, they can easily be recognized because s or some mul- 
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tiple of s will be common to all terms in eq. (12). For example, if a n = 
in eq. (12), s would be common to all terms and could be placed outside 
the brackets. 

Step 2. Examine eq. (12) to see that all the coefficients of s are non- 
zero and of the same sign. If this is not true, an unstable system is im- 
mediately indicated. 

Step 3. Arrange the coefficients in an array of the form: 



Index 



n 


a 


02 


04 


«6 


n — 1 


ai 


a s 


05 


07 


n-2 


h 


b 2 


h 




n — 3 


C\ 


c 2 


C3 




n — 4 


dx 


d 2 






n — 5 


ei 


e 2 






n - 6 


A 








n-7 


9l 


• 







The index number indicates the highest order of s in a row. 
The first two rows consist of all the terms in the given equation and the 
rest are calculated in the following fashion. 



&i = 



ci = 



rfi = 



etc. 



a 1 a 2 


— 


a a 3 




ai 




Ma 


- 


aj6 2 




h 




Cl&2 


— 


&lC 2 



Cl 



aia 4 — a a 5 
b 2 = 



c 2 = 



d 2 = 





«1 




, etu 


&1«5 


- 


aih 


etc 




h 




Cl&3 


- 


hc 3 


etc. 



Cl 



Notice that two terms in the first column are used in each calculation. 
As the term to be calculated shifts to the right, the additional two terms 
in the formula shift to the right also. The formulas for calculation of 
terms in a row use only those terms in the two rows immediately above. 
The process is continued for (n + 1) rows where n is the order of the charac- 
teristic equation. 

Step 4- After the array has been completed, stability can be investigated 
by inspection of terms in the first column. The number of changes in sign 
of the terms in the first column is the number of roots with positive real parts. 
This constitutes Routh's criterion. 



STABILITY 21-07 

Example. Given the fourth order equation 

8s 4 + 2s 3 + 3s 2 + s + 5 = 0. 
The array becomes : 
Index 
4: +835 



+ 2 



- 1 



I"* 2-3-8-1 ~ 2 il 

[\ 2-5-8-0 2-5 i 
[h = g = -2- = 5 \ 

r -l-i -2-5 -11 n 

L Cl= — -i — = ^r = 11 J 

i. a-11 [* H-5 -(-1-0) 11-5 I 
1: +11 |* = n = -fi- = 5 J 

0: +5 

There are two changes of sign in column one (between indexes 3 and 2, 
and 2 and 1), therefore the equation must have two roots with positive 
real parts. Since a fourth order equation has four roots, the remaining two 
roots must lie in the left half s-plane. 

Note. A generalization can be made from this example. The last term 
(+5) came down through the array without change. Since all the coef- 
ficients in the equation are positive, the first two terms in column one are 
positive. Only terms of index 2 and 1 in column one can be negative. 
Thus, a maximum of two sign changes can occur. Therefore, one can 
conclude that if all terms in a fourth order equation are nonzero and of the 
same sign, at least two roots must lie in the left half s-plane. This con- 
clusion is of no great import in itself but it merely points the way to 
intelligent use of this method of analysis. 

Special Cases in Applying the Routh Criterion. Because the Routh 
criterion can be used to advantage in other commonly used stability studies, 
it is worth while to pursue the criterion in greater detail here. 

a. Row multiplication. Any row may be multiplied by a positive con- 
stant without affecting the criterion. This may be used to decrease the 
arithmetic labor involved. 

b. When the first term in a row is zero and other terms in the same row are 
not zero. To continue the process, replace the first column zero by an arbi- 
trarily small positive constant, A, and continue the calculations. Examine 
the complete array in the usual fashion. If necessary, A may be assigned 
any arbitrarily small value. This number may be positive or negative 
but is customarily assumed positive. 
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c. When all terms in a row are zero. This special case arises when roots 
lying radially opposite and equidistant from the origin occur as shown in 
Fig. 3. A pair of conjugate pure imaginary roots is of this category. When 
a row of zeros occurs, take the preceding row of coefficients and form a 
subsidiary function. This subsidiary function is the polynomial in s 
having as coefficients the terms of a row; the ex- 
ponent of the highest power of s is the index of the 
row and successive powers of s decrease by two. 

Example. The subsidiary function of the row 
with index 3 of the preceding example is 



X- 



JO) 



U 



I s-plane 



Fig. 3. Roots radially 

opposite and equidistant 

from origin. 



/(s) = 2s 3 + s, 

whereas the subsidiary function of the row with 
index 2 is 

/(s) = -s 2 + 5. 



Upon formation of the subsidiary function of the row preceding the 
row of zeros, differentiate it with respect to s and replace the row of zeros 
by the corresponding coefficients of the differentiated function. Proceed 
in the usual manner. The index numbers remain unaltered. Upon com- 
pletion of the array, the number of changes in sign indicates the number 
of roots in the right half s-plane. The remaining roots are either in the 
left half s-plane or on the axis of imaginaries. One of several procedures 
can be utilized to determine the number of each (Ref. 4). A straightfor- 
ward approach is as follows. 

In the original equation replace s by — s. This substitution rotates all 
the roots of the equation through 180 degrees. Those roots of the original 
equation in the left half s-plane are now in the right half s-plane. Applica- 
tion of Routh's criterion to this new equation determines the number of 
these roots. Thus, the number of roots of the original equation in the 
left half s-plane has been ascertained. The total number of roots is equal 
to the order of the original equation. Therefore the number of roots on 
the axis is equal to the total minus the sum of those in the right and left 
half s-planes. 

Relative Merits of Routh's Criterion. This criterion serves as a 
quick check on absolute system stability. It can also be used to advantage 
in the more powerful Nyquist criterion. It nicely avoids the necessity 
for factoring an equation to determine the nature of its roots. This method 
does not provide a clear indication of system performance and does not 
clearly show the ways to improve a design should improvement be required. 
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4. NYQUIST STABILITY CRITERION 

This powerful criterion is based on the fact that the frequency response 
of the open loop transfer function indicates the stability characteristics of 
the closed loop system. In Fig. 1 the open loop transfer function is repre- 
sented by G(s)H(s). 

Restrictions on the General Nyquist Criterion. 

a. G(s)H(s) must be the ratio of the transforms of linear differential 
equations. 

b. G(s)H(s) must be single valued and an analytic function (Ref. 5) 
for all values of s having zero or positive real parts except at possible discrete 
points (Ref. 4). 

Basic Definitions. In general G(s)H(s) is a fraction of rational poly- 
nomials in s. 



(13) G(s) = 

(14) H(s) = 



Di(s) (s + s 2 )(s + s 4 )(s + s 6 ) 
N 2 (s) Kzis + s'Jis + s'a) • 



D 2 (s) (s + s' 2 )(s + sU)(s + s' 6 ) 
The all important eq. (2) can be written as 

tfi(«)iV a (a) 



(15) 1 + G(s)H(s) = 1 + 

(16) 1 + G(s)H(s) = 



Di(s)Z) 2 («) 

Di(s)D 2 (s) + Nds)N 2 (8) 
Di(8)D 2 (s) 



Characteristic Function. [Di(s)D 2 (s) + Ni(s)N 2 (s)] represents the charac- 
teristic function of the closed loop system of Fig. 1. The characteristic 
equation is merely the characteristic function set equal to zero. 

Zeros. The factors (s + s{), (s + s 3 ), •••, represented by iVi(s) are 
called zeros of G(s). This terminology arises because when s takes on the 
value of a root of Ni(s), i.e., — si, — s 3 , • • •, iVi(s) equals zero and G(s) does 
likewise per eq. (13). 

Poles. The factors (s + s 2 ), (s + s 4 ), •••, represented by D\(s) are 
called poles of G(s). When s takes on the value of a root of Di(s), i.e., 
— s 2 , — s 4 , • • •, Di(s) equals zero and G(s) goes to infinity per eq. (13). 
This rise to infinity is called a pole. 

Note. Per eq. (16), poles of G(s)H(s) are also poles of [1 + G(s)H(s)] 
whereas zeros of [1 + G(s)H(s)] are unknown and their nature to be 
determined by the stability criterion. Zeros of [1 -f- G(s)H(s)] are poles 
of C(s)/R(s). 
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Nyquist Criterion. General Procedure. 

a. Plot G(s)H(s) for s traversing the boundary of the entire right half 
s-plane in a clockwise direction. (See following note.) 

b. Draw a vector, V, from (—1 + jO) [the minus one point in the 
6r(s)#(s)-plane] to G(s)H(s) and observe the angular rotation of this 
vector for the above values of s. 

c. Let R be the net number of revolutions of this vector. R is positive 
for counterclockwise revolutions and negative for clockwise revolutions. 

d. Determine the number of poles of G(s)H(s) in the right half s-plane, 
i.e., poles with positive real parts. Call this integer number P. If necessary, 
Routh's criterion may be used to determine this. 

e. The number of zeros of [1 + G(s)H(s)], Z, is determined from the 
equation 



(17) 



Z = P - R. 



f. The system is stable if and only if Z = 0, i.e., if the number of 
counterclockwise revolutions of G(s)H(s) about the —1 point is equal to 
the number of poles of G(s)H(s) in the right half s-plane. 

Note. If G(s)H(s) has any poles on the jco-axis (i!e., pure imaginary 
roots), when s is taking on values up the jco-axis, it must bypass these 
points. It is customary to make s traverse a small semicircle to the right 
of these points as shown in Fig. 4. If G(s)H(s) ever does have poles on 



+jco 
jdi'-T) Out at oo- 



-to:) 



: >- 



s-plane 



-7«? 



Fig. 4. Traversal of s for the Nyquist plot where G(s)H(s) has poles at dtjui and 0. 



the jco-axis whose values are unknown, it is almost as much effort to de- 
termine these as it is to find the zeros of [1 + G(s)H(s)] directly. In this 
case, Dzung's criteria (Refs. 19, 20) may be a better approach for stability 
analysis. Fortunately this condition arises infrequently. 

The Physical Meaning of Making s Traverse the jco- Axis. In short, 
it is obtaining the steady-state frequency response of the open loop transfer 
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function G(s)H(s). Consider the case shown in Fig. 5. A sin wt is the 
input and in the steady-state condition, B sin (at + 6) is the output. To 
be theoretically exact, the steady-state condition is the condition that exists 
after an infinite time has elapsed. This allows all the transients to die 
out to absolute zero for a stable system. For practical consideration, 
steady state occurs after the transients have settled down to arbitrarily 



A sin at 



G(s)H(s) 



B sin (ut + 6) 



Fig. 5. Frequency response measurements. 

small values. Comparison of the ratio of the output sinusoid to the input 
sinusoid reveals that a gain change, B/A, and a phase shift, 9, have oc- 
curred. This gain change and phase shift are due to G(s)H(s) and can be 
considered as the magnitude and direction of a vector. This is the vector 
notation of the steady-state behavior of G(s)H(s). 

Sinusoidal Input Variable. The Laplace transform of the input 
variable is 

(18) £ _1 [sin co/] = 



+ co 2 (s-f- ja)(s - ju) 

The graphical representation of the Laplace transform of this sinusoid 
is a pair of points on the imaginary axis a distance of ±co from the origin. 
As the frequency of the sinusoid varies, co varies with it. See Fig. 6. 

Consider the case where 

(19) G(s)H(s) = K • 

(s + s 2 ){s -f s 4 ) 

The poles of G(s)H(s) are plotted at — s 2 , and — s 4 in the s-plane in 
Fig. 6. In this same figure are plotted the poles of the input sinusoid 
whose frequencies are successively co , coi, co 2 , co 3 , co 4 , • ••, co n . The cor- 
responding vectors representing the gain change and phase shift G(jca)H(joo) 
for each frequency are plotted in Fig. 7. One method to obtain G(s)H(s) 
for any particular s is to substitute the particular value of s in eq. (19). 
An equivalent, but more illuminating, procedure is to consider each factor 
in eq. (19) as a separate vector. In Fig. 6 are shown the two factor vectors 
for s = jw 2 . As s assumes values up the jco-axis, the vectors from the roots 
increase in magnitude and phase. Since these vectors appear in the 
denominator of eq. (19), as s traverses up the jco-axis, the magnitude of 
G(s)H(s) decreases whereas its phase becomes increasingly negative. For 
this particular transform given by eq. (19), for positive co, G(jui)H(ju>) lies 
in the third and fourth quadrants. The entire curve expands or contracts 
with respective increase or decrease in the gain constant K. 
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Fig. 6. s-Plane plot of input sinusoid and 
vectors of G(s)H(s) factors. 



Fig. 7. G(s)#(s)-Plane plot of 
frequency response olG(s)H(s). 



Conformal Mapping. Mathematically, G(s)H(s) is a function which 
transforms a point in the s-plane to a point in the G f (s)/f(s)-plane. This 
mapping of points or curves in one plane to points or curves in another 
plane is called conformal mapping. The line along the jco-axis in the 
s-plane maps into the curve in the G(s)H(s) -plane shown in Fig. 7 by use 
of the transform G(s)H(s). An important point to remember is that any 
curve in the s-plane produces a corresponding curve in the G(s)H(s)- 
plane. The curve in the s-plane which lies on the joj-axis corresponds to 
the input function being a variable frequency sinusoid. The shape of 
the corresponding curve in the G(s)H(s) -plane depends on the particular 
fraction of polynomials represented by G(s)H(s). 

Figure 8 shows other lines along which s might vary. Figure 9 shows 
the corresponding curves of G(s)H(s) for G(s)H(s) given by eq. (19). 
Points on line (1) correspond to input functions of the form. 



(20) 

whose Laplace transform is 

(21) JB-^e-*!* sin at] = 



e ^ sin oit 



(s + <ri + jw)(s + tri - ju) 
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Points on line (2) correspond to input functions whose Laplace transform 



is 
(22) 



cfi-V 1 sincof] = 



(s - o-i + ju)(s - o-i - ju) 



The conformal mapping procedure obtains definite corresponding curves 
for G(s)H(s) as shown in Fig. 9. 



Imaginary 



■(Ti + JQO 





(<Tl + M) 



G(s)H(s) -plane 



Fig. 8. Particular paths of s in 
s-plane. 



Fig. 9. 



Plot of G(s)H(s) for paths of s 
in Fig. 8. 



Principles of Nyquist Criterion. By use of conformal mapping princi- 
ples it can be shown (Ref. 6) that if s is made to traverse the boundaries 
of a given area, observation of the behavior of the vector from the —1 
point to G(s)H(s) in the G(s)i7(s)-plane indicates how many zeros of 
[1 -f- G(s)H(s)] lie in the area whose boundaries were traversed by s. 

Refer to Fig. 10, where s is made to traverse the boundary of area A, 
and the corresponding path of G(s)H(s) is as shown in Fig. 11. Observa- 
tion of the net rotation of the vectior V about the — 1 point gives a clear 
indication of the roots of [1 + G(s)H(s)] in area A. For every pole of 
G(s)H(s) located in area A, V will experience one net counterclockwise 
rotation about the — 1 point. For every zero of [1 + G(s)H(s)] in area 
A, V will experience one net clockwise rotation about the — 1 point. There- 
fore if the number of poles, P, of G(s)H(s) in area A is known, the number 
of zeros of [1 + G(s)H(s)] in area A can be found by subtracting from P 
the number of net revolutions of V about the — 1 point. If area A is 
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made to encompass the entire right half of the s-plane, existence of zeros 
of [1 + G(s)H(s)] in this area can be determined from the above procedure 
and stability of the closed loop system can be ascertained! 



Imaginary 




Fig. 10. Arbitrary path of s in 
the s-plane. 



G(s)H(s)-p\ane 



Fig. 11. Corresponding path of 
G(s)H(s). 



The left portion of the boundary in Fig. 12 corresponds to making the 
input to G(s)H(s) a sinusoid. The traversal out at infinity is only of 
mathematical importance because infinite values are difficult to handle 
in physical equipment. For practical purposes, that finite region relatively 

close to the origin is of major im- 
portance as will be more clearly 
demonstrated in the Bode approach 
to stability analysis. 

Summary. Stability is uniquely 
defined by those values of s which 
make 

1 + G(s)H(s) = 0. 

To ascertain existence of zeros of 
[1 + G(s)H(s)} in the right half s- 
plane, Nyquist's criterion requires 
s to traverse the boundary of the 
entire right half s-plane. The por- 
tion of the boundary of major im- 
portance, thejco-axis, corresponds to a sinusoidal input function. Therefore, 
the frequency response of the open loop transfer function G(s)H(s) gives 
clear indication of stability of the closed loop system. This is most for- 
tunate because constant amplitude variable frequency generators are 
much easier to build than exponentially varying variable frequency 
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Fig. 12. 



Path of s enclosing entire right 
half s-plane. 
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generators, 
merited. 



Experimental procedures are thereby more easily imple- 



Application of Nyquist Stability Criterion. 

Example 1. Given 

K(s + Sl ) 



G(s)H(s) = 



s(« + s 2 )(s + s 4 )(s + s 6 ) 



In Fig. 13 consider s in the region from b to c. As co becomes increasingly 
large, R 

lim [G(s)H(s)] =- = /-270 . 



In this region G(s)H(s) approaches zero asymptotically to the — 270- 
degree direction, i.e., the +jj3-line. As s traverses the boundary c-d-e 

Imaginary 



-X — X— O— X i'r-jh 



ja 



s-plane 



>d 




Fig. 13. s-PIano plot 
r ,, m , K(s+ Sl ) 



Fig. 14. Nyquist plot of G(s)H(s). 



s(s + s 2 )(s + Si)(s + 



out at infinity, the G(s)H(s) vector rotates 540 degrees in the counter- 
clockwise direction, but since the magnitude is zero, this rotation is un- 
observable. The region e-f is the conjugate of c to b. In the region / 
to g there is a continuous curve which wiggles a bit because of the pole 
zero locations as shown in Fig. 13. At point g the s traverse takes a 90- 
degree turn to the right. In conformal mapping, angles are preserved in 
the small, therefore the G(s)H(s) plot also takes a 90-degree turn to the 
right. In the region g-h-a, G(s)H(s) behaves like K/s. 

K 

lim [G(s)H(s)] = — 

s-+0 S 
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In other words, the movement of s is very close to the pole at the origin 
so the vectors of the poles, and zeros relatively far away do not experience 
great change. The vector of the pole at the origin experiences a 180- 
degree change in the counterclockwise sense. Since this vector is in the 
denominator of G(s)H(s), G(s)H(s) experiences a 180-degree change in 
the clockwise sense. 

The region a to b is the conjugate of g to /. The G(s)H(s) plot is usually 
plotted solid for s = -\-joi and dotted for the rest of the boundary. From 
Fig. 13 it is apparent that G(s)H(s) has no poles in the right half s-plane. 
P = 0. Notice that the zero of G(s)H(s) is not considered at all. If the 
gain constant is such that the — 1 is at point A in Fig. 14, R = 0. There- 
fore 

Z = P-R = 0-0 = 0, 

and the closed loop system is stable. 

If the gain constant is raised such that the — 1 point is at B, there are 
two clockwise encirclements of the — 1 point, 

Z = P - R = 0- (-2) = +2, 

and the closed loop system is unstable and has two poles in the right half 
s-plane. 

If the gain were adjusted such that the —1 point were at D, i.e., the 
G(s)H(s) curve passes right through the —1 point, R is indeterminant. 
This condition produces a constant amplitude sinusoidal oscillation in the 
closed loop system. A change in the gain constant is like changing the 
calibration on the coordinate axes. 

Example 2. Given 

K(s - 10) 

s 2 + 100 
In the region a to b in Fig. 15 

K(-10+ja) kVu 2 -f- 100 

G(s)H(s) = — ~ = — /tan-W-10) . 

100 - co 2 100 - co 2 L -^ ~ 

K 

For co = 0, G(s)H(s) = — /-180°. 

10 ' 

As co increases, the magnitude of G(s)H(s) increases and the phase angle 
becomes more negative. 

As co approaches 10, G(s)H(s) approaches infinity along the +135-degree 
line. In the region b-c-d 

\im[G(s)H(s)]= K ■ 
s-Vio (s - J 10) 
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Therefore, since s takes a 90-degree right turn and proceeds 180 degrees 
counterclockwise, G(s)H(s) takes a right turn and proceeds 180 degrees 
clockwise. In the region d-e, G(s)H(s) is well behaved and proceeds to 
zero as s approaches joo. As s — > «>, 

K 

lim [G(s)H(s)] = - = /-90° . 

s—*jao S 

Along e-f-g, G(s)H(s) remains at zero. The rest of the curve is the con- 
jugate image of e to a. For this system P = 0. From Fig. 16, R = 
for the —1 point at A. This system is stable. 

For the —1 point at B, R = —1 and Z = 1. This system is unstable. 



Imaginary 



JO) 

d 



;iox\ 



-yiof) 



£* 



10 



s-plane 




Fig. 15. s-Plane plot 

K(s - 10) 



Fig. 16. Nyquist plot of G(s)H(s). 



G(s)H(s) = 



s 2 + 100 



Practical Considerations in Plotting Diagrams. If one should ever 
find that the number of counterclockwise encirclements of the —1 point 
is greater than P, he may correctly infer that he has made a mistake in 
calculating either P or R ! 

The procedure in drawing the Nyquist diagrams is first to draw in the 
approximate shape of the G(s)H(s) curve for the prescribed traversal of 
s. The labor involved is by no means negligible. To avoid unnecessary 
labor, the reader is advised to learn first how to use the following Bode 
diagrams and apply them to obtain the exact Nyquist plot when necessary. 
The Bode approach is presented after the Nyquist criterion for ease in 
presenting the requisite theory. Subsequent usage should by no means 
be affected by order of theoretical presentation. 
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Strictly speaking, the small semicircles about poles of G(s)H(s) on the 
jco-axis and also the traversal of s out at infinity do not correspond to 
constant amplitude sinusoidal input. The polar plot used in the Nyquist 
criterion is therefore not strictly a frequency response plot. For purposes 
of simple definition, these exceptions are overlooked. 

Abbreviated Nyquist Stability Criterion. When the open loop trans- 
fer function is stable by itself 

P = 0, 

and the criterion for stability reduces to 

R = 0. 

Statement 1. For a stable open loop transfer function, the closed loop 
system will be stable if there are no encirclements of the — 1 point in 
the G{s)H{s)-plane for s = jw. 

The criterion may be further reduced to observing the behavior of 
G(s)H(s) for positive co in the region where the magnitude of G(s)H(s) is 
near unity. The additional restriction is that G(s)H(s) becomes a constant 
less than 1 (or zero) as s becomes increasingly large. This restriction 
means that in eq. (15) the order of N\(s)N 2 (s) is less than or equal to the 
order of D 1 (s)D 2 (s). Where the respective orders are equal, the product of 
gain constants, K\K 2 , from eqs. (13) and (14) must be less than 1. 

For the cases that fall within the above-mentioned restrictions (and 
there are many), the criterion can be restated. 

Statement 2. In the region of frequencies where G(jo))H(joi) is near the 
unit circle, the system is stable if the — 1 point is not encircled. 

Statement 3. If the further restriction is imposed that G(jo))H(jo)) is 
well behaved in the region of the unit circle, then stability is indicated by the 
phase angle of G(ju)H(jui) for positive values of u when it crosses the unit 
circle. For phase angles less than —180 degrees at unit circle crossover 
the system is stable. For phase angles more negative than — 180 degrees 
at unit circle crossover, the system is unstable. In Fig. 17, Gi(s)Hi(s) 
represents a stable closed loop system whereas G2(s)H 2 (s) represents an 
unstable system. A well-behaved G(s)H(s) is loosely defined as one that 
does not wander too much in the region of the unit circle. A not too well- 
behaved open loop transfer function is shown in Fig. 18. For systems of 
this type, the general Nyquist criterion should be used. Adequate in- 
formation about system stability is contained in Fig. 18, but more than 
the first unit circle crossover must be inspected. 

Phase Margin. For those systems that do fall within the abbreviated 
criterion, additional definitions have evolved. 

The phase of G(ju>)H(ju>), measured with respect to the positive real axis 



STABILITY 



21-19 



and defined as positive in the counterclockwise sense, is given as 0. The 
phase margin is the phase of G(ja))H(ju) at unit circle crossover and is 
measured with respect to the direction of the — 1 point : 



(23) 



7 = 180° + 0. 



In Fig. 17, Gi(s)Hi(s) has a positive phase margin whereas G2(s)H 2 (s) 
has a negative phase margin. Phase margin at unit circle crossover evidences 
system stability with plus and minus values indicating stable and unstable 
systems respectively. Zero phase margin at unit circle crossover means that 
G(ja))H(ja)) passes through the —1 point and therefore that the closed 
loop system will sustain a constant amplitude oscillation. 
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Fig. 17. Unit circle in the G(s)i/(s)-plane. Fig. 18. Not too well-behaved G(s)H(s). 

Example. In Fig. 18 the phase margin at the first unit circle crossover 
is positive. The —1 point is encircled so the system is unstable. This 
example illustrates the case where inspection of only the first unit circle 
crossover could lead to erroneous conclusions. 

Gain Margin. A second point of particular significance is the gain or 
magnitude of G(jco)H(jo)) where it crosses the negative real axis. This is 
o"i in Fig. 19. The reciprocal of this value is the gain margin of the system. 
The gain constant of G(s)H(s) could be raised by a value l/o-i before 
instability arose. 

The — 1 point can be considered a vector of unit magnitude and direction 
of — 180 degrees. Note that the phase margin is defined with relation 
to the magnitude of the — 1 point whereas the gain margin is defined with 
relation to the direction of the — 1 point. 

Conditional Stability, Unconditional Stability. A conditionally 
stable system is one Avhere instability can come about by either an increase 
or decrease in system gain. An unconditionally stable system is one where 
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Fig. 19.. , Determination of phase margin, gain margin. 

instability can come about only for an increase in system gain. Figures 20 
and 21 illustrate these cases. 
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Fig. 20. Conditionally stable system: 
K(s + si)(s + s 3 ) 
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Fig. 21. Unconditionally stable system: 
K 



G(s)H(s) = 



s(s + s 2 )(s + s 4 ) 



Inverse Polar Plots. The preceding Nyquist diagrams are polar plots 
of G(s)H(s). These diagrams led to ascertainment of the nature of the 
zeros of eq. (2), 

1 + G(s)H(s) = 0. 
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If in this equation both sides are divided by G(s)H(s), 



(24) 



1 



0(8)H(8) 



+ 1=0. 



Let G'( s )#'( s ) represent the inverse of G(s)H(s) 



(25) 



G'(s)H'(s) + 1 = 0. 



The above mathematical manipulations cannot alter the factors of eq. 
(2). The zeros of eq. (24) or (25) are exactly the same zeros of eq. (2). 
Investigation of system stability via the inverse polar plot leads to con- 
clusions identical to those arrived at by use of the direct polar plot of 
G(s)H(s). In certain design applications use of the inverse loop transfer 
function may more clearly demonstrate effects of design changes. 

Polar Plots of Some Common Open Loop Transfer Functions. 
The following plots represent some commonly encountered system func- 
tions. Once the reader recognizes how these were generated, he should be 
ready to handle any newly encountered situation. In the G(s)H(s) -plane 
plots are the letters A, B, C. These represent possible locations of the — 1 
point dependent upon the value of the gain constant K. Stability is 
indicated for various locations of the — 1 point. See Figs. 22-36. See also 
Figs. 13-16. 
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Fig. 29. Polar plot of 
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Multiple Input, Multiloop Systems. A system of this category is 
shown in Fig. 37. 

Stability characteristics of any linear system are independent of where 
the input or output functions are located. All closed loop transfer func- 
tions of a given system have the same characteristic equation. There- 
fore, when merely investigating stability, one can select one input and 
one output and proceed to reduce the multiloop system to the basic form 
shown in Fig. 1. Stability analysis proceeds in the aforementioned fashion. 



Ri(s) + 



sat® 




Fig. 37. Multiple input, multiloop system. 



It is important not to mislead the reader into believing that transfer 
characteristics are independent of input and output locations. If Ri(s) is 
a reference command and i^ 2 (s) is a corruption disturbance, it is very 
possible to design the system so that Ri(s) is accepted, whereas Zt^s) 
is rejected. In essence, this is the purpose of system design as will be shown 
in the following chapters. 

Relative Merits of the Nyquist Criterion. This powerful criterion 
reveals the stability characteristics of practically all linear systems. It 
also presents a clear indication of ways to improve the system design should 
this be desired. System design and compensation will be discussed fully 
in Chap. 23. 

Systems defined by linear differential equations whose coefficients are 
time variant cannot be handled directly by this method. Fortunately, the 
greatest number of systems encountered in practice are not of this category. 

The one major disadvantage of this criterion is the large amount of 
labor involved in drawing the polar plots or Nyquist diagrams. When a 
complete detailed stability analysis is required, the following Bode diagrams 
should be used to aid in construction of the requisite polar plots. 
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5. BODE ATTENUATION DIAGRAM APPROACH 

This stability analysis procedure is exactly the same as that used in the 
abbreviated Nyquist criterion except that the information is examined in 
a .different fashion. The information presentation is changed in a way 
which allows design modification without the large amount of labor at- 
tendant in drawing the Nyquist diagrams. 

The Nyquist complex plane diagrams (see Sect. 4) contain three pieces 
of information. The first pertains to the path or contour that the com- 
plex variable, s, traversed. The second and third pieces of information 
consist of the magnitude and phase of the transform, G(s)H(s), for each 
value of s. When s traversed the axis of imaginaries, it took on values of 
real frequency, o). The corresponding plot of G(jco)H(ju) was calibrated 
with respect to co. See Figs. 6 and 7. 

The same information contained in Fig. 7 can be conveyed by means 
of two diagrams with co a common parameter as in Fig. 38. Because of 



\G(ju)H(ju)\ 



Magnitude 



/G(ja)H(j<a) 



Phase 



03 > 

Frequency Frequency 

Fig. 38. Bode attenuation diagrams, frequency response curves. 

the fundamental work done by Bode (Refs. 7 and 8) in this field, these 
diagrams are commonly called Bode diagrams. Since these diagrams 
provide information only for s on the ^"co-axis, they are true frequency 
response curves. 

Fundamentals of Bode Diagrams. Consider first the logarithm of a 
complex number G(ju>). 



(26) GU<*)=P+tf 

(27) log G(ja) = log M(u) + j[0(co) + 2xn], 



n = 0, 1, 2, 3, 



M (co) is the magnitude of the complex number (or vector) and 0(w) is 
its phase angle. 2-jrn occurs because an integer number of revolutions 
causes the vector G(ju) to lie in the same place. The principal value of 
the logarithm is 



(28) 



logG0'«) = logM(co) +jd(a). 
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If G(jui) consists of the product of two complex numbers, Gi(jui) and 
G 2 (jw), the logarithm becomes 

(29) log G 1 (ja)G 2 (ja) = log M !(«) + log M 2 (u) + j[*i(«) + 2 («)]. 

In other words, the logarithm of the product of two complex numbers- is 
the sum of the individual magnitude logarithms plus j times the sum of 
the separate phase angles. Note that the logarithm of a complex number 
is another complex number. Therefore, in adding complex numbers, real 
parts add to real parts and imaginary parts add to imaginary parts. 

Equation (29) shows that by introducing the logarithm concept, the 
difficult process of multiplication is replaced by the simpler process of 
addition. 

Equation (28) shows that the log-magnitude [log ilf (co)] and the phase 
[0(a)] are separate functions of the common parameter co. They can be 
plotted separately as in Fig. 38. For minimum phase networks (Ref. 9), 
the magnitude and phase are so uniquely related (Refs. 7 and 8) that 
specification of one also specifies the other. The definition of a minimum 
phase network can be complex (Ref. 9), but a simple "not-all-inclusive" 
definition is given here. A minimum phase network is one where for a speci- 
fied magnitude characteristic, the phase is the minimum possible at all fre- 
quencies. In short, the transform of a minimum phase network is one with 
no poles or zeros in the right half s-plane. 

Bode Diagram Stability Analysis. Consider those well-behaved 
transfer functions discussed for the abbreviated Nyquist criterion and 
shown in Fig. 17. Stability was uniquely related to the phase angle at 
unit circle crossover or to the magnitude of G(jui)H(jw) when its phase 
angle was —180 degrees. To repeat, for well-behaved, minimum phase 
G(s)H(s), the negative feedback, closed loop system is stable if the phase angle 
at unit circle crossover is less negative than —180 degrees or whose magnitude 
is less than unity when the phase angle is — 180 degrees. 

To use this important information, it is necessary only to transform 
the unit circle and the negative real axis into the frequency response dia- 
grams. Whenever G(jut)H(jco) lies on the unit circle, its magnitude is 
equal to 1 regardless of its phase or the frequency involved. The logarithm 
of this magnitude is zero; therefore the unit circle in the Gr(s)i7(s)-plane 
corresponds to the line where log M = in the log magnitude plot. 

In terms of the decibel concept, the unit circle corresponds to the zero- 
decibel line, 

Whenever G(ju>)H(ju>) lies on the negative real axis, its phase is —180°. 
Therefore the negative real axis in the G(s)H(s) -plane corresponds to the 
— 180-degree line in the phase plot. These important stability landmarks 
are shown by heavy lines in Fig. 39. 
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Fig. 39. Minimum phase response curve regions. 

The Nyquist Stability Criterion Rephrased in Terms of the Bode 
Diagrams. For well-behaved, minimum phase G(s)H(s), the closed loop 
system is stable if at the frequency where the log magnitude of G(jo))H(jui) is 
equal to zero, its phase angle is less than — 180 degrees. Or, the system is 
stable if at the frequency where the phase angle of G(jco)H(jui) is —180°, 
the log magnitude is less than zero. If the condition arises where the phase 
angle is equal to — 180 degrees and the log magnitude is zero at a frequency 
o) , the closed loop will sustain a constant amplitude oscillation at a fre- 
quency <o . This condition corresponds to G(ju)H(jui) passing through 
the — 1 point in the Nyquist plot. 

If the phase angle of G(jo))H(jco) is defined in terms of phase margin as 
given by eq. (23) and Fig. 19, the stability criterion is commonly expressed 
as follows. At gain crossover (the point where the magnitude curve crosses 
the log M = line) a positive phase margin indicates a stable system whereas 
a negative phase margin indicates an unstable system. 

By use of Bode diagrams it is possible to deduce whether a system is 
or is not stable in situations more complicated than that wherein G(s)H(s) 
is a well-behaved, minimum phase network. When complicated situations 
arise, final conclusions should be checked by use of the general Nyquist 
criterion or the Routh criterion. 

Mechanics of Drawing Bode Diagrams. When given a transform 
G(s)H(s), the most straightforward procedure in drawing Bode diagrams 
is to pick values of jca, substitute into G(jco)H(jui), and grind out the 
complex algebra. Fortunately this laborious procedure is not required 
frequently because G(s)H(s) is usually known in factored form. There 
are four basic building blocks used in drawing Bode diagrams. 

1. K ±l , a pure gain constant. 

2. s ±x , a pure differentiation or pure integration. 

3. (s -f- coo)^ 1 , a simple lead or simple lag. 

4. (s 2 -f- 2fo) s -+- coo 2 )^, a quadratic lead or quadratic lag. 

In reverting these basic factors to logarithmic plots it would be entirely 
possible to use logarithms to the base e and to use the common multiplier 
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of 1. Since the decibel concept was in vogue and orders of 10 are easier 
to handle than orders of e, logarithms to the base 10 were used and the 
multiplying factor was taken as 20. A decibel is equal to 



(30) 



Pq Vo 
Decibels = db = 10 logio — = 20 logio 

i i ' i 



Transfer functions in general are more similar to voltage ratios, V /V{, 
than to power ratios, Po/P{, therefore the multiplying factor of 20 is 
commonly used. Some writers (Ref. 10) would rather use the multiplying 
factor of 10 and units of decilogs, but this seems to be of small consequence 
in stability analysis. 

The First Building Block: The Pure Gain Constant. 



(31) 



20 log K ±l = ±20 log K. 



The logarithm of a pure gain constant is independent of frequency and 
therefore plots as a horizontal line in the magnitude and phase curves. 
K has zero phase angle if it is positive and — 180 degrees if it is negative. 



Magnitude 



S»S 





e -90 e 

-180° 



^20)ogK(K>\) 
-p-20 log #(#<!) 



Phase 



x ^-* /iC(+ number) 
y^ -/K{- n u m ber) 



log co- 



Fig. 40. Bode diagram of a pure gain constant. 



It might appear here that logarithms of negative real numbers exist. Note 
that a purely real logarithm of a negative number is an impossibility, 
but a complex logarithm of a complex number is possible and given by 
eqs. (27) and (28). A pure gain constant is a complex number with zero 
imaginary part. Bode plots of pure gain constants are shown in Fig. 40. 
Note. Log co is plotted at the bottom of the figure and is common to 
both curves. 
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The Second Building Block: Pure Differentiation or Pure In- 
tegration. 



(32) 



20 log s ±1 1 = ±20 log co ± jv/2. 



Equation (32) shows that a pure differentiation, s, introduces a constant 
phase angle of +90 degrees whereas the pure integration, 1/s, introduces 
a constant phase angle of —90 degrees. 

Log co increases by a factor of 1 for every decade change in co; therefore 
the log-magnitude changes by 20 db per decade change in co. If the log- 
magnitude is plotted to a linear scale and co is plotted to a logarithmic 
scale, the magnitude curves for s ±l consist of straight lines with ±20 db 
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Fig. 41. Bode diagram of s^ 1 factor. 



per decade slopes. Since log 2 is approximately equal to 0.3 and 20 X 0.3 
= 6, the slope is also commonly referred to as 6 db per octave. These 
curves are shown in Fig. 41. Semilog paper lends itself very nicely to 
this application. Notice that the curves pass through db at co = 1. 
This occurs because 



(33) 



20 log 1 = 0. 
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The Third Building Block : Simple Lead or Simple Lag. This factor 
is put in nondimensional form by multiplying by w /w . 



(34) 



(s + coo)* 1 = - (« + coo)* 1 = coo* 1 (- + l) 
coo \wo / 



±1 



Since w is a constant, it can be handled as a pure gain factor. The 
logarithm of the frequency variant part becomes 



(35) 20 



og(-+l) 

\co / 



= ±20 log 



(") 

\w / 



+ 1 +jtan- 1 — • 

.COn/ 0)q 



Consider first the magnitude expression 



(36) M(co) = ±20 log J[—) + 1 = ±10 log 



(-) 

Awn/ 



-1+1 

WQ/ 



Substitution of values of w in this expression leads to the log magnitude 
curves drawn in Fig. 42. The curves are drawn for the simple lag. To 
obtain values for the simple lead, reverse the sign of the ordinate values. 
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The straight lines are the asymptotes to the log magnitude curves. For 
co much less than co , co/coo is much less than 1 and can be neglected in eq. 
(36). Therefore 

M(u) I « ±10 log 1 = db/decade. 

(37) I »«»„ 

For co much greater than co , co/co is much greater than 1. Therefore 
from eq. (36) 

(38) M(w)\ « ±20 log — • 

This is similar to the expression for a pure differentiation or pure in- 
tegration and leads to a slope of ±20 db per decade change in the non- 
dimensional parameter co/co . The point where the asymptote curves meet 
is at co/coo = 1 and since the slope breaks from db per decade to ±20 
db per decade at this frequency, co is called the break frequency. Since the 
junction resembles a corner, it is also called a corner frequency. Note 
that a pure differentiation or integration corresponds to a factor with a 
break frequency at co = 0; therefore all that can be seen on a Bode diagram 
for this factor is the ±20 db per decade slope. 

At co/coo = 1 the exact curve lies ±3 db away from the asymptote curves. 
At co /co = 2 or }4, the correction is a ±1 db. Normally, it is sufficient 
to plot in the asymptote lines, the three corrections at co/co = 3^> 1, 2 
and sketch in the curve freehand. 

From eq. (35) the phase angle of a simple lead or lag is 

(39) 0(«) = ± tan -1 — 

co 

For co = 0(co) = 0°, 

co = OO 0(co) = ±90°, 

co = coo 0(w) = ±45°. 

The phase angle curve can be calculated from eq. (39) and is shown in 
Fig. 42. To simplify the labor involved, templates can be cut to the 
0(co) shape and used to draw in the phase curve for simple lead or lag 
factors (commonly referred to as simple breaks). Caution must be exer- 
cised in that each template is good only for a given calibration on the log co 
and 0(co) axes. 

The servomechanisms scale shown in Fig. 43 can also be used to obtain 
the phase angle of simple breaks. The 45-degree arrowhead is placed at 
the frequency at which the phase is desired, and the phase is read from the 
scale at the break frequency. A scale, like a template, is good only in 
conjunction with the calibration on the log co axis for which it was designed. 
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Fig. 43. Plastic servomechanisms scale. (Developed at the General Electric Company 
by J. J. Hogle and W. E. Sollecito.) 

The plastic scale shown in Fig. 43 has scales on the top and botton sides 
for 3-cycle and 4-cycle semilog paper for a particular calibration on the 
logarithmic axis. 

The Fourth Building Block : A Quadratic Lead or Quadratic Lag. 

This factor (s 2 + 2^coqs + co 2 ) ±1 , in nondimensional form is 



±1 



(40) 201og — + 2r-+l 



\«o 



wo 



= ±20 log 



i 



2fco\ 2 / co 2 \ 2 

■' + ( 1 -^) ± ' tan "' 



co 






2fto co 

2 2~' 

COq — CO 



As evidenced in eq. (40), the magnitude and phase functions are de- 
pendent not only on co but also on the damping ratio, f . For a given damp- 
ing ratio, there is only one magnitude and one phase curve. The quadratic 
factor 



<coo 



+ 2r— +1 

COQ / 



is plotted in Figs. 44 and 45 as a family of curves with f an independent 
parameter. For 

/ s 2 s V 

(— + 2r-+l) 

Vcon Wn / 



+1 



the curves would merely be inverted because all values would take on 
opposite sign as shown by eq. (40). 

The magnitude asymptote lines, shown as heavy dashed lines in Fig. 
44 meet at co/co = 1. This co is referred to as a complex or quadratic 
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break frequency. From eq. (40) 



(41) MM^OlosjQV^-J) 2 . 

For co/coo much less than 1, all co/co terms are negligible and 



(42) M(co) 



±20 log Vl = db/decade. 



-«i 



For co/coo much greater than 1, the dominant term is 
(43) M(u) 



\\ coo 2 / 



-»i -±201ogJ(-— ) =±40 log ~ 



This is merely twice the slope of a simple break for the similar assump- 
tion. Therefore, the asymptote of a complex break is ±40 db per decade 
for large co/co . 

When a quadratic factor is encountered, a first approximation is made 
by considering f = 1, which means that a simple break of multiplicity 
2 occurs at co - For more accurate work, the data in Figs. 44 and 45 must 
be used. In this case, f is calculated from the given quadratic factor and 
the requisite magnitude and phase information obtained from the cor- 
responding £" curves. 

The scale shown in Fig. 43 can be used to obtain phase of quadratic 
factors by proper use of the additional scales on the right and left sides. 
Since a graph must be kept for magnitude information, it seems logical 
to use a graph for phase information also. 

It is possible to plot graphs of magnitude correction terms for simple 
and complex factors to an expanded scale to improve accuracy. Since 
the corresponding phase correction curves offer small expanded scale 
possibilities, it is usually of little value. 

The Transform of a Pure Time Delay e~ sT . This is of particular 
interest in many cases. 

( 44 ) 20 log e~ sT I = 20 log 1 - j a T = - juT. 

Equation (44) shows that the magnitude is independent of frequency 
and the phase is linearly related to frequency. The magnitude and phase 
curves are shown in Fig. 46. This function falls within the limitations 
of the Nyquist criterion, and stability can be investigated in the usual 
fashion. 
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Fig. 46. Bode diagram of e~ sT . 
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Application of Bode Diagrams. 

Example 1. Draw the Bode diagram of 



G(s)H(s) = 



316 



s(s + 10) 



First put the simple lag factor in nondimensional form 



G(s)H(s) = 



31.6 



a[(«/10) + 1] 
Separate individual factors 

G(s)H(s) = 31.6 • - • * • 

s (s/10) + 1 

The asymptotic approximate and the exact Bode diagrams of these in- 
dividual factors are shown in Fig. 47. The composite G(s)H(s), in heavy 
solid lines, is merely the summation of all the separate magnitude and 
phase curves as indicated by eq. (29). At gain crossover co = 16, 6 = — 147° 
for the exact curve whereas co= 18 and 6 = —150° for the approximate 
curve. In most cases the approximate answers are sufficiently accurate 
because in practice the transfer functions represent average values and 
will not correspond exactly with delivered equipment. Also, as equip- 
ment wears in normal use, the transfer characteristics change. For these 
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reasons the designer must usually provide a margin of safety and some 
adjustments which will permit small changes when required for improved 
system performance. 

The system shown in Fig. 47 is stable for all values of loop gain because 
the phase angle approaches —180° asymptotically. The Nyquist plot of 
a similar transfer function is shown in Fig. 25. It is well to keep both 
representations in mind. 




0.1 0.2 0.4 12 4 10 20 40 100 200 400 1000 

co >- 

Fig. 47. Bode diagram of G(s)H(s) = 31.6 • (1/s) • [l/(s/10) + 1]. 

Example 2. Given the following loop transfer function, determine K 
such that gain crossover occurs at 6 = — 135° at w greater than 100 rad/sec. 



G(s)H(s) = 



K(s 4- 80) 



(s 2 + 6s + 100)(s + 400) 



Again, by nondimensionalizing and separating factors 

G(s)H(s) 

K-80 1 



100-400 [(s/10) 2 4- (0.6s/10) + 1] 



a-) 



(s/400) + 1 



The frequency response curves are shown in Fig. 48. The composite 
curves can be drawn without resort to drawing all the individual curves 
as done in Fig. 47. Neglect the constant term and consider first the 
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asymptote approximations to the separate factors. There is a quadratic 
lag break at 10, a simple lead break at 80 and a simple lag break at 400. 
The approximate curve is flat out to 10, breaks down to — 40 db per decade 
at 10, breaks to — 20 db per decade at 80 because of the simple lead, and 
then breaks back to — 40 db per decade at 400 and continues on at this 
slope. The servomechanism scale shown in Fig. 43 is very useful in drawing 
these asymptote lines. The exact curve is drawn in by obtaining correction 
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10 20 40 100 200 400 1000 2000 4000 10,000 

to — > 

K 



(s/80) + 1 



500 [(s/10) 2 + (0.6s/10) + 1] [(s/400) + 1] 



terms for the quadratic lag for f = 0.3 from Fig. 44 and for the simple 
lead and simple lag from Fig. 42. The exact phase curve is drawn in by 
use of the servomechanism scale shown in Fig. 43 for the simple breaks 
and the phase curve for the quadratic lag by use of the phase curve for 
f = 0.3 in Fig. 45. The arrowhead on the servomechanism scale is placed 
at the frequency where the phase is desired, and the phase contribution 
by the simple breaks is read at the break frequencies. The lead and lag 
terms contribute positive and negative phase angles respectively. 

To set the gain constant such that gain crossover occurs at 6 = — 135° 
at co greater than 100, the entire magnitude curve is shifted up until 
this occurs. Instead of shifting the magnitude curve up, it is simpler to 
shift the zero db line down. This corresponds to recalibration of the db 
axis. The required amount of 0-db line shift corresponds to K/500. To 
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meet the requirements of the example K/500 = 42 db. K therefore must 
equal 63,000. 

Use of Bode Diagrams in Drawing Nyquist Plots. When system 
design is attempted by use of the Nyquist diagrams, it soon becomes ap- 
parent that the labor involved in drawing the diagrams is excessive. This 
comes about because design changes come in terms of multiplying factors 
which are laborious to incorporate because multiplication is a relatively 
complex process. The logarithm concept of the Bode diagrams reduces 

10 
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12 4 10 20 40 100 

0) >■ 

Fig. 49. Reproduction of magnitude of Fig. 47. 

multiplication to the simple process of addition. It is advantageous first 
to plot a Bode diagram and transfer the values from this diagram to the 
polar plot when information is desired in such form. The major stumbling 
block to this procedure is the conversion of decibels to gain numbers. It 
is possible to plot the Bode magnitude diagrams on log-log paper, as shown 
in Fig. 49 and thereby circumvent the use of decibels. Gain factors are 
clearly brought to view. 

This approach suffers from two major disadvantages. First, the plot 
of phase is still best accomplished on semilog paper, therefore separate 
scales would be required for the magnitude and phase curves. Use of 
decibels allows a single semilog paper to be used for both plots. Second, 
in adding the magnitudes of two factors, a pair of dividers or some such 
device would become necessary. Shift of the zero line is not as simple as 
it is with the decibel scale. 

A more useful approach is to use a scale as shown in Fig. 50. The scale 
is transparent. Three possible scale factors on the decibel scale are avail- 
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Fig. 50. Gain-decibel conversion scale. 
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able. The scale is placed vertically on the Bode magnitude diagram and 
values of gain read directly. 

Another approach is that of a graph as shown in Fig. 51. Values of 
decibels are read off the magnitude curve and the graph is used to convert 
decibels to gain numbers. 
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Fig. 51. Gain-decibel conversion chart. 



Relative Merits of the Bode Diagram Approach. This approach 
is by far the most generally used method in system design. Design modi- 
fications can be analyzed with a minimum of labor involved in drawing 
the diagrams. The approximate curves allow the designer to investigate 
a host of designs in a short time. The most promising approximate designs 
are then investigated more exactly. The host of curves provides an indica- 
tion of how system performance will change in the presence of some non- 
linearities in the system. 

This approach is based on the limitations of the abbreviated Nyquist 
criterion, and when complex situations arise, it is best to revert to the 
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Nyquist criterion or Routh criterion for exact stability evaluation. The 
Bode diagrams can be used to draw the Nyquist diagrams. 

6. ROOT LOCUS METHOD 

This method (Refs. 11-13), developed by Evans, provides a means for 
obtaining the roots of the characteristic equation of a closed loop system, 
the values of which clearly indicate system stability. Essentially, the 
method assumes that a chosen complex number is a root of the charac- 
teristic equation and tests to see if it can be. If this test is favorable, one 
constant is changed to a value such that the complex number is a root of 
the equation; This constant is the loop gain of the closed loop system. 

The complex numbers which represent possible roots of the characteristic 
equation, when plotted in the s-plane and identified with the necessary 
corresponding loop gain, form curves which are the loci of the characteristic 
equation roots. These roots represent poles of the closed loop response 
which clearly indicate system stability and transient performance. 

This plot in the s-plane provides a rapid evaluation of the effects of 
varying the gain in a system. It provides a graphical representation of 
the predominant features of a closed loop system, i.e., its poles, and when 
system behavior is inadequate, provides a clear indication of proper 
compensation. 

For a system whose loop transfer function is given by 



G(s)H(s) = 



K 



s(T 2 s+l)(T 4 s+l) 
the root locus plot is shown in Fig. 52. This plot shows that as the loop 




Fig. 52. Root locus for 



G(s)H(s) = 



K 



s(T 2 s + l)(T iS + 1) 
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gain, K, is increased, the closed loop poles move in the direction of the 
arrowheads. For all values of gain less than K c , the closed loop poles lie 
in the left half s-plane, and the system is stable. For all values of gain 
greater than K c , the system is unstable. K c is a critical gain factor. Also 
shown by the plot is the fact that, as the gain varies from K i to K c , the 
closed loop poles have a decreasing damping factor. Therefore, one can 
expect the transient response to be more oscillatory and to have a longer 
settling time as the gain is increased in this region. 

Theory of Construction. Consider the general negative feedback 
system shown in Fig. 1. 

Assuming that the feed forward and feedback transfer functions are 
composed of fractions of rational polynomials in s, i.e., 

a mi s m i + a^.iS™! -1 H \- a^ + a N^s) 

( 45 ) G ( s ) = , .. ■ , IZ=I = ' 



(46) H(s) = 



b ni s n i + b ni _ lS n I" 1 + • • • + M + 6 £i(s) 
d m2 s m * + d^-is" 1 *" 1 + • • • + d lS + d N 2 (s) 



(47) 



e n2 s n * + e„ 2 _!S n 2 1 -\ h eis + e Q D 2 (s) 

The closed loop response is 

C(«) 0(8) iVi(8)D 2 (s) 



R(s) 1 + G(s)H(s) D 1 (s)D 2 (s) + iV^A^s) 



The root locus method obtains the roots of the fractional equation 
[1 + G(s)H(s) = 0], which are the roots of the characteristic function 
[Di(s)D 2 (s) + Ni(s)N 2 (s)]. It is of interest to note that the closed loop 
response has numerator factors (zeros) which are identical to the zeros of 
the feed forward transfer function and poles of the feedback transfer 
function, N\(s) and D 2 (s) respectively. 

To find the closed loop poles 

1 + G(s)H(s) = 0. 
Therefore 

(48). G(s)H(s) = -1 = 1 /±Nt , N = 1, 3, 5, 7, • • •. 

For this identity to exist, the angle of G(s)H(s) must lie along the negative 
real axis of the s-plane. This constitutes the angle condition: 

(49) /G(s)H(s) = ±Nt, N = 1, 3, 5, 7, • • •. 

Also, the magnitude of G(s)H(s) must be unity. This constitutes the 
magnitude condition: 

(50) \G(s)H(s)\=l. 
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In general, 

(51) G(s)H(s) = 

where 

(52) 
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K(S + 8{)(S + S 3 )(S + 8 B ) • • • (S + S 2 w-l) 
(S + S 2 )(s + S 4 )(s + S 6 ) •••(«+ S 2n ) 

^(Sl)(«3)(«5) • • ' (S 2m -i) 
(S 2 )(S 4 )(S 6 ) • • • (s 2 ») 



represents the loop gain. Each factor in eq. (51) represents a vector in the 
s-plane as shown in Fig. 53. 



Vector (s + si) 




Vector s 
S" Vector si 

5 



s-plane 



Fig. 53. Vector representation of typical polynomial factors. 

The angle condition, eq. (49), requires that 
(53) 

[/s+st +/s+s 3 -\ +/s+s 2m _i] - [/s+s 2 -\ +/s+s 2n ] = ±Nt 



or 

(54) 


k—m i=n 
E /(S + S 2fc _i) - E /(« + S2.0 = ±NlT. 



This states that for an exploratory point, s, to lie on the root locus, the 
summation of the angles of the zeros minus the summation of the angles 
of the poles of the open loop response must be an odd multiple of ir. ' 

Procedure. The procedure is to plot the poles and zeros of G(s)H(s) in 
the s-plane, choose an exploratory point, s, lay off the factor vectors [note 
that the factor vector arrowheads always lie on the exploratory point s 
in Fig. 53], sum the angles with the proper sense; if they add up to ±iW, 
the point is on the root locus. If not, move the exploratory point over and 
repeat. This constitutes assuming that a chosen complex number is a 
root of the characteristic equation and testing to see if it can be, i.e., if it 
satisfies the angle condition. 
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When a point is located that does satisfy the angle condition, then the 
vector magnitudes are measured and values are substituted in the magni- 
tude condition, eq. (50), to calibrate the constant K. [Note. K determines 
loop gain, but it is not defined as such per eq. (52).] 

,„, |(s + s 2 )| |(s + s 4 )| ••■ |(s + s 2 „)| 

(55) |A| = 



(s + Si) | | (s + s 3 ) I • • • I ( s + s 2m -i) I 

Repetition of the above steps should ascertain the complete locus. 

When the locus is completed, the actual K of the given system is de- 
termined from G(s)H(s). By reference to the root locus plot, the closed 
loop poles are then obtained by inspection. 

General Theorems for Construction. At first glance, it appears as 
though a root locus may lie anywhere in the complex plane and to dis- 
cover it may be a hit-or-miss proposition. Fortunately, the locus must 
take on certain definite patterns governed by the number and location of 
the open loop poles and zeros. The following general theorems aid in 
ascertaining the approximate root locus. 

Theorem 1. Number of branches of the locus is equal to the number of 
closed loop poles. A branch is a separate portion of the root locus which 
has all values of loop gain on it. For a given loop gain only one pole may 
exist on one branch of the complete root locus plot. The number of 
branches is therefore equal to the degree of the characteristic equation in 
s because this determines the total number of poles. Reference to Fig. 54 
shows that for a given loop gain, K\, there are four closed loop poles, one 
on each of the four branches labeled (1), (2), (3), (4). K is the loop gain 
because all factors are in nondimensional form. 

Theorem 2. The locus starts at open loop poles or infinity (K = 0) and 
ends at open loop zeros or infinity (K = =o). Inspection of the magnitude 
eq. (55) shows that at open loop poles K is zero because one of the nu- 
merator magnitudes becomes zero. K is infinite at open loop zeros be- 
cause a zero magnitude term appears in the denominator. For the locus 
to start at infinity it is imperative that G(s)H(s) have more zeros than 
poles, i.e., its numerator would be of higher degree than its denominator. 
Equation (55) shows that for this case and for s approaching infinity, K 
approaches zero. For the locus to end at infinity, it is imperative that 
G(s)H(s) have more poles than zeros. In this case eq. (55) shows that for 
s approaching infinity K approaches infinity also. Figure 54 shows the 
case Where the loop transfer function has four poles and one zero. The 
branches start at the poles for a loop gain of zero. As the loop gain in- 
creases to infinity, branch (2) goes along the real axis from the pole to the 
zero while the other three branches tend toward infinity. 
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Theorem 3. For the locus to exist on the real axis, the sum of poles and 
zeros to the right of the exploratory point must be odd. This is so because 
conjugate complex roots together contribute zero angle when the ex- 
ploratory point is on the real axis. Only poles and zeros on the real axis 
to the right of the exploratory point contribute angle (180° each), there- 
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Fig. 54. Root locus plot for 
G(s)H(s) = 
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fore, the above conclusion. By again referring to Fig. 54 it is seen that 
where the locus exists on the real axis there are either five or three poles 
and zeros to the right. 

Theorem 4. The locus is symmetrical with respect to the real axis. The 
characteristic equation is a rational polynomial in s with real coefficients. 
Therefore, the roots, when complex, must occur as conjugate pairs. In 
Fig. 54, branch (3) is the image of branch (4) about the real axis. 

Theorem 5. The locus leaves an open loop pole or approaches an open 
loop zero in the direction given by ±IV7r minus the sum of angles of vectors 
from remaining poles and zeros to the pole or zero in question. Consider the 
exploratory point s to be very close to an open loop pole. As s circles the 
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pole, the angle change due to the vector from the pole in question to s 
changes greatly. The other vectors change direction only minutely. 
Therefore, since the angle contribution from all other poles and zeros is 
nearly fixed, the angle contribution of the pole vector in question must 



Asymptote line /I 




Asymptote line 
4 



/ 



/-/ 1 



-15 -12 -9/Wv-5 ^3 




-12 
Centroid 

p ( -3 )4+ 6-9-l 2 = _ 7 ] 



Probable 
locus 



Direction of departure 

[180° - 90° + 54' - 33* - 24° = 87°) 

;4 



s-plane 



Fig. 55. Construction theorems 
G(s)H(s) = 



K(l + 1) 



G +1 )(n+0fe+0(^i+0 

contribute the amount necessary to satisfy the angle condition. There- 
fore, the direction of locus departure from an open loop pole is ascertained. 
A similar situation arises near an open loop zero. Reference to Fig. 55 
shows that the direction of departure of the locus from the upper complex 
pole is 87°. 

Theorem 6. The direction of locus asymptote lines is given by 



±Ntt 



m 



n = number of poles, 
m = number of zeros. 
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When the exploratory point is extremely far from the cluster of open 
loop poles and zeros, they all contribute essentially the same amount of 
angle. Since these must add up to ±180 degrees or some odd multiple, 
the foregoing conclusion exists. In Fig. 55 it is seen that ±60° and 180° 
are the directions of the asymptote lines. Part of the 180-degree line 
also happens to be a branch of the locus. 

Theorem 7. Asymptote lines cross real axis at 

i—n k=m 



k= 

n — m 



2^, <*i — 2-i ff k 

i=i k=i a = real part of ith poles, 

ak = real part of fcth zero. 



This corresponds to the centroid of the pole-zero cluster. In Fig. 55 
the asymptote lines cross the axis at —7. 

Practical Considerations. If the loop transfer function has many 
poles and zeros, some of which are located relatively far from the main 
cluster and from the jco-axis, a first order approximation can be made to 
the exact locus by omitting the distant poles and zeros. This procedure 
requires good engineering judgment. The advantage lies in quicker 
ascertainment of the important part of the locus which can be drawn to 
an expanded scale. 

The procedure to be used in drawing a root locus is to plot the poles 
and zeros of the open loop response. From the preceding generalizations, 
sketch in the loci. Graphically determine the exact loci. With the open 
loop gain constant pick off the closed loop poles. The closed loop response 
is then made up of the poles obtained from the plot and the zeros and 
multiplying constant from inspection of G(s) and H(s) per eq. (47). Multi- 
ple loops are handled by first reducing the minor loops to transfer functions 
in factored form. It is of interest to remember here that the root locus 
method is a graphical procedure of factoring the characteristic equation 
of a system. The minor loop transfer functions are then included as blocks 
in the major loop and the major loop root locus is then plotted. 

Donahue's Analytical Procedure to Calculate the Root Loci. A 
relatively simple analytical means of plotting a root locus has been de- 
veloped (Ref. 34). This method determines a point on the locus by shift- 
ing the jco-axis a given distance, a\, and then calculating the frequency, 
wi, at which the locus crosses the jco-axis in the s-plane. The requisite 
loop gain is then calculated from a>i. Successive points on the locus are 
obtained by successive shifting of the ./co-axis. Tables 1 and 2 have been 
derived by Donahue to aid in the calculations (Ref. 34). 
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Referring to the general single loop negative feedback system of Fig. 1, 

let 

K(R N + I N ) 
(56) G(s)H(s) = K \ 

Kd -r Id 

where Rn is the real part of the numerator (even powers of s) and In is 
the imaginary part of the numerator (odd powers of s). The denominator 
follows similar notation. 

For a point to lie on the root locus, from eq. (48), 

K(R N + I„) 

(57) ~l^TT^=- 1 - 

Therefore 

(58) KR N + KI N = -R D - Id. 
Equating real part to real part and imaginary to imaginary 

, . ^ Rd Id 

(59) K=- =- 

K N 1 N 

and 

(60) RdIn = RnId. 

Equations (59) and (60) provide means of solution for frequency and 
gain at the root locus crossing of the jo-axis. 
Example. Let 

(61) G(s) = -± ^ H(s) = 



(62) G(s)H(s) = 



(s + s 2 ) (s + s 4 ) 

K(s + Sl ) K(s + h ) 



(s + s 2 )(s + s 4 ) s 2 + 7?is -f r 
where h = s 1} r-i = s 2 + s 4 , r = S2S4 

from the preceding 

R N = h , I N = s, R D = s 2 + r , I D = ns. 

Substitution in eqs. (59) and (60) gives 

(s 2 + r ) 

(63) K= - K -—-^> 

h 

(64) (s 2 + r )s = ho^s. 
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Table 1. Aid for Root Locus Cal 



(m,ri) 


General Form 


do 


a\ 


(1,2) 


K(s+h ) 
(s 2 +ns+r ) 


ro+ritr+cr 2 


ri+2a 


(2,2) 


K(s 2 +h lS +ho) 
(s 2 +ris+r ) 


ro+ri<r+<r 2 


ri+2a 


(0,3) 


K 


ro+ria-+r2(T 2 +(T i 


ri+2r 2 <r+3cr 2 


(s 3 +r2S 2 +riS+ro) 


(1,3) 


K(s+h ) 


ro+ria +r 2 <r 2 +a 3 


ri+2r 2< r+3cr 2 


(s z +r2S 2 +ris+rd) 


(2,3) 


K(s 2 +h lS +h ) 
(s 3 +r2S 2 +ris+r ) 


ro+ncr+r2(T 2 +<T 3 


r 1 +2r 2 <T+3<r 2 


(3,3) 


K(s 5 +h 2 s 2 +his+h ) 
(s s +r2S 2 +riS+ro) 


ro+ri<y+r2a 2 +a 3 


ri+2r 2 ff+3(7 2 


(0,4) 


K 


ro+na+r2(T 2 

+r 3 <r 3 +<r 4 


ri+2r2cr 

+3r 3 <7 2 +4(7 3 


(s 4 +rss 3 +T2S 2 +ris +ro) 


(1,4) 


K(s+h) 


ro+n<r+r2<r 2 

+r 3 <7 3 +<7 4 


ri+2r2cr 

+3r 3 <r 2 +4<x 3 


(s 4 +r3S 3 +r2S 2 +ris +ro) 


(2,4) 


Z(s 2 +ft lS +M 


ro+ri<T+r2cr 2 

+r 3 <r 3 +* i 


/•i+2r2ff 

+3r 3 <r 2 +4(r 3 


(s 4 +r3S 3 +ros 2 +ris +ro) 


(0,5) 


K 


ro+rio-+r2cr 2 

+r3(r' i +r4a i +<T b 


ri+2r2o-+3r3o- 2 

+4r 4 (7 3 +5(7 4 


(s 5 +r4S 4 +r3S 3 +r2S 2 -\-r\S +ro) 


(1,5) 


X(s+/i ) 


ro+n<r+r2(T 2 

+r3<r 3 +r4a i +a 5 


ri+2r-2<r+3r 3 <r 2 

+4r 4 a 3 +5(T 4 


(s 6 -f r 4 s 4 +?"3S 3 +r2S 2 +ris +ro) 


(0,6) 


K 


ro+ri(r+r 2 o- 2 +r 3 cr 3 
+n<r i +r h a h +a 6 


ri+2r2o-+3r3o- 2 
+4r 4 a 3 +5r 5 <r 4 +6(T 5 


(s 6 +r5S 5 +r 4 s 4 +r 3 s 3 +r2S 2 +ris+ro) 



(m,n) 



U=o J 



[,§-'] [,§'-"-"] 
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CULATION BY DONAHUE PROCEDURE 

ct2 az a 4 a$ 







r2+3cr 



















bo 


6i 


&2 


h +<T 








lo+hiff 


hi+2<r 





W 







r2+3ff 



h +<x 



r2+3er 







r 2 +3<r 

r2+3r 3 o-+6<r 2 r 3 +4<r 



ho+hi<r h+2a- 

W 

h +hi<r hi +2/i 2 (T /i 2 +3<r 

+/i 2 <r 2 +<r 3 +3<r 2 





r2+3r30-+6<r 2 r3+4<r 



ho+v 



r2+3r3<r+6<T 2 r3+4er 



/io -\-h\a- h\-\-2<r 

+* 2 



r 2 +3r 3 <r 

■ c 2 , in 3 ^3+4r 4 (r r 4 +5a 

+ 6r 4 . 2 + 10. 3 +1Q(t2 







r2+3n<x 



rs +4r 4 o- 



+6r 4 <r 2 + 10<r 3 +10ff 2 



r 4 +5cr /lo+ff 



r2+3r3<r+6r4<r 2 r3+4r 4 <r r 4 +5r5<r r5 

+ 10r 5 <r 3 + 15<r 4 + 10cr 2 +20<r 3 +15o- 2 +6<r 



[£(£*) 

Li=2 \fc=l / 







E^l [Z [z(Zfc) 

i=o J Li=i _ Li=2 \it=i / 
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In eqs. (63) and (64) let s = ju 

co 2 — r 

(65) K = — 

h 

(66) co 2 = r - rjio. 

Equations (65) and (66) define the gain and frequency at which the root 
locus crosses the imaginary axis. 

To calculate other points on the locus, shift the ju-axis by replacing 
s with (s -f- a) 

K(s + a + h ) 

(67) G(s + a)H(s + a) = , ' , , , ' ' 

(s + a) z + r^s + a) + r 

which reduces to 

K(s + 6 ) 

(68) G(s + a)H(s + a) = —± — . 

s + dis + a 

where 

do = r + ar x + o- 2 , a x = r t + 2o-, 6 = h -f tr. 

By analogy to eqs. (62), (65), and (66) 

(69) ~ K = T- (flo ~ ^), 

6 

(70) o? = a — ai&o- 

Tables for Donahue Procedure. This example gives rise to the first 
row of Tables 1 and 2. For a = eqs. (69) and (70) reduce to eqs. (65) 
and (66). Table 1 has the parameters a , a\, a 2 , etc., and b , b\, b 2 , etc., 
determined in terms of the original numerator and denominator power 
series coefficients for each of several types of loop transfer functions. 
Table 2 gives w 2 and —K in terms of these a and b parameters. 

The procedure therefore consists of reducing G(s)H(s) to a fraction of 
two power series, identifying this with the proper row in Table 1, sub- 
stituting in values of a, which lead to calculation of the parameters a 
and b and subsequent solution of co 2 and — K per Table 2. a 1} coi, and K\ 
provide a point on the root locus. The occasion may arise that for a given 
a, there may exist no real co or positive K. This merely signifies that no 
locus exists in this portion of the s-plane. 

It will be noted that the 1 last line in Table 1 has the general equation 
which can be used to evaluate the a's and b's for additional transfer func- 
tions. But since the corresponding general equations for the K and co 2 
are missing, the above serves more as a check on new derivations than as 
a means of avoiding work. 



(m, n) 


(1,2) 


(2,2) 


(0,3) 


(1,3) 


(2,3) 


(3,3) 


(0,4) 


(1,4) 


(2,4) 


(0,5) 


(1,5) 



Table 2. Aid for Root Locus Calculation by Donahue Procedure 

co 2 -K 

— [<Zo — CO 2 ] 

[a — co 2 ] 



[ao - 


6 oi] 


[Mi - 


- Mo] 


[«i - 


-6i] 
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Construction Aids 

From the discussion in the preceding subsections, it may be inferred 
that locating points that satisfy the angle condition is a time-consuming 
procedure. To aid in this respect, a simple device can be constructed which 
mechanically sums the vector angles. 

Mechanical Angle Summer. (See Fig. 56.) This device is made of 
clear plastic. The arm rotates on the disk with a slight drag. To use, place 
the pin point at an exploratory point s with arm pointing horizontally to 
the left and the zero degree arrowhead aligned under the arm centerline. 



Arm 



X 



Diagonal 



Centerline 



v- 



Top View 




Disk 

Pin head 
Pin point 



Arm 



X 



^Pin head 
\/ .•-Disk 



'^ — ■" Pin point 
Side View 
Fig. 56. Mechanical angle summer. 

To sum angles of pole vectors, hold the disk in place and rotate arm center- 
line to a pole root. Release disk and return arm to neutral. Friction causes 
disk to rotate with arm. To sum angles of zero vectors, reverse the order 
of disk rotation. Rotate arm centerline to a zero root (disk free to 
rotate). Hold disk and return arm to neutral. When all roots have been 
successively accounted for and the arm has been returned to the neutral 
position, the 180-degree arrowhead should lie under the arm centerline for 
a point to be on the root locus. 

The Spirule shown in Fig. 57 (Ref. 13) performs the above operation 
plus the additional feature of calibrating the locus. A logarithmic spiral 
curve on the arm permits the logarithm of a length to be obtained as an 




Fig. 57. Spirule. (Developed by W. Evans. Available from the Spirule Company, 

Whittier, Calif.) 
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angle, so that the addition of such angles corresponds to adding logarithms. 
The root locus is calibrated rather simply with this addition. 

Conductive Paper Disk. (See Figs. 58 and 59.) To minimize further 
the labor involved and therefore enhance its use, machines have been 
devised which perform the necessary operations automatically. One 
such machine uses the fundamental idea that the electric potential de- 



Lines of equipotential 




Electrodes 

Conducting 
paper disk 



Fig. 58. Angle measurement on a conductive paper disk. 



A-c 
supply 




V::log R/R 



Electrodes 
Conductive paper 
Circles of equipotential 

Fig. 59. Magnitude measurement on a conductive paper disk. 

veloped on a conducting paper disk could represent angles or logarithms 
of lengths. This principle has been tried with success (Ref. 35). 

A Mechanical Plotting Machine. This machine, described in Ref. 
14, is a simple mechanical instrument which sums angles of vectors by 
using the principle that torque developed by a rotational spring is pro- 
portional to the angle of rotation of the spring. The machine is simple 
in construction, portable, and it requires no auxiliary power. 

A Compact Analog Machine. Described in Ref. 15 and called the 
"Complex Plane Analyzer" this machine can, among other functions, be 
used to obtain a root locus plot. The principle involved is that of reducing 
vector multiplication to two independent summations of phase and log 
magnitude. To this end, a logarithmic potentiometer is used to measure 
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magnitude and a linear one measures phase. Capacitors are individually 
charged with voltages representing these quantities. Summation of 
capacitor voltages produces the required overall products and quotients. 
The machine is simple, rugged, can also be used to plot phase loci, and is 
available commercially. 

Some Common Root Loci 

The following plots (Figs. 60-91) are presented to aid in checking some 
of the preceding theorems, to present some general loci and to show in 
general how redistribution or variation in number of poles and zeros 
affects the plot. 

jw JO) 



s- plane 



-s 4 -s x -s 2 



Fig. 60. Root locus plot for 
K(s + Sl ) 



G(s)H(s) = 



(s + s 2 )(s + s 4 ) 
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Fig. 61. Root locus plot for 
K(s + s 2 ) 
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Fig. 62. Root locus plot for 
G(s)H(s) 



(s +a +j0)(s +a -j/3) 




Fig. 63. Root locus plot for 

G(s)H(s) 

K(s + si) 



(s+a+i/3)(*+a -]0) 
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Fig. 64. Root locus plot for Fig. 65. Root locus plot for 
K(s + si) K 
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Fig. 66. Root locus plot for 

G(s)H(s) 
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Fig. 67. Root locus plot for 

G(s)H(s) 
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Fig. 68. Root locus plot for 
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Fig. 69. Root locus plot for 
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Fig. 70. Root locus plot for 

G(s)H(s) 

K(s + 8l) 
(.s + s 2 )(s + s 3 )(s + s 4 ) 




Fig. 71. Root locus plot for 

G(s)H(s) 

K(s + si) 

(S + S 2 )(S + S 3 )(S + Si) 
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Fig. 72. Root locus plot for 

G(s)H(s) 
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Fig. 73. Root locus plot for 

G(s)H(s) 
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Fig. 74. Root locus plot for 
K(s + Sl ) 



G(s)F( S ) 
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Fig. 75. Root locus plot for 
K(s + Sl ) 



(S + S 2 )(S + S 3 )(S + Si) 



G(s)H(s) = 
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Fig. 76. Root locus plot for 

K(s + Sl )(s + s 3 ) 
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Fig. 77. Root locus plot for 

K(s + Sl )(s + s 3 ) 
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Fig. 78. Root locus plot for 

K(s + si)(s + s 3 ) 



G(s)H(s) = 



s(s + s 2 )(s + s 4 ) 




Fig. 79. Root locus plot for 

K(s + Sl )(s + s 3 ) 



G(s)H(s) = 



s(s + s 2 )(s + s 4 ) 



STABILITY 



21-65 




jot 



Fig. 80. Root locus plot for 
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Fig. 81. Root locus plot for 
K(s + Sl )(s + s 3 ) 
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Fig. 82. Root locus plot for 
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Fig. 83. Root locus plot for 

G(s)H(s) 
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Fig. 84. Root locus plot for 
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Fig. 85. Root locus plot for 

G(s)H(s) 
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Fig. 86. Root locus plot for 
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Fig. 87. Root locus plot for 
G(s)H(s) 
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Fig. 88. Root locus plot for 
K(s + si) 
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Fig. 89. Root locus plot for 

G(s)H(s) 
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Fig. 90. Root locus plot for 
G(s)H(s) = Ke- Ts . 




Fig. 91. Root locus plot for 
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Interpretation of Results 

The root locus plot provides a pictorial representation of the roots of 
the characteristic equation of the closed loop response. The location of 
these roots determines the modes of the transient response. Figure 92 
shows contours of constant characteristics of these modes. 



Line of 
(constant fwo 



Jb> ^-Line of constant cod 



Circle of constant wq 




Fig. 92. Contours of constant characteristics of transient response modes. 

The jco-axis defines the limit of absolute stability. For the system to 
be absolutely stable, all roots must lie in the left half s-plane. Circles 
concentric with the origin correspond to loci of roots with constant un- 
damped natural frequency. Therefore, for a system prescribed to have a 
maximum natural frequency mode, all roots must lie within the cor- 
responding prescribed circle. Lines of constant imaginary part correspond 
to lines of constant damped natural frequency. For prescribed maximum 
damped natural frequency, all roots must lie within the area bounded 




Fig. 93. Location of roots for combined restraints. 

by the corresponding prescribed lines of constant imaginary part. Lines 
of constant real part correspond to lines of constant response time or con- 
stant exponential decay factor ( — fco ). Again for prescribed maximum 
individual response time, all roots must lie to the left of the corresponding 
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line of constant negative real part. Radial lines passing through the 
origin correspond to lines of constant damping ratio (f). For prescribed 
minimum damping ratio, all roots must lie within the area bounded by the 
corresponding minimum damping ratio lines encompassing the negative 
real axis. Note that lines of zero damping factor, infinite response time, 
and absolute stability are the same. 

Combined restraints may be imposed on the modes of the transient 
response by reducing the area of the root location to that area common 
to the individual areas. For example, with specified maximum response 
time, maximum damped natural frequency, and minimum damping ratio, 
the roots would have to lie within the cross-hatched area of Fig. 93. 

Multiloop Systems Analysis 

For multiloop systems, the procedure is to reduce the individual inner 
loops to transfer functions in factored form by use of minor loop root loci. 
The major root locus is then drawn up as a single loop. A particular ad- 
vantage of the root locus method of analysis is that, when changes are made 
in the minor loops, the effect on the overall loop is shown directly. 

For example, consider a closed loop voltage regulating system shown 
in Fig. 94. When a load is imposed upon the system, the gains change 
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Fig. 94. Multiloop voltage regulating system. 



because of nonlinear behavior. The major loop gain decreases whereas 
the minor loop gain increases. 

The minor loop root locus is shown in Fig. 95. 

The major loop root is shown in Fig. 96. This figure reveals that the 
net effect on the overall system of imposition of full load is that the domi- 
nant pole pair (those complex roots closest to the origin) shifts to a lower 
frequency with a slightly higher damping ratio whereas the subdominant 
pole pair (those complex roots furthest from the origin) shifts to a higher 
frequency with a lower damping ratio. The conclusion here is that imposi- 
tion or removal of load does not severely affect the system stability or 
performance. 
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Fig. 96. Major loop root locus plot. 



STABILITY 21-71 

In multiloop systems, desired performance of the overall loop can some- 
times be achieved by use of unstable minor loops. In these instances, it 
must be remembered that if a failure can occur such that the remaining 
system releases large amounts of uncontrolled energy, the design should 
be critically reviewed. In practice, systems are usually designed with 
stable inner loops. 

System Design 

By nature, synthesis is more complicated than analysis. A few general 
observations can be made with regard to reshaping the root locus to obtain 
the required root locations. Inspection of the plots shown in Figs. 60 to 91 
shows that poles tend to repel the locus whereas zeros tend to attract 
it. Also, as the difference between the number of poles and zeros increases, 
the locus tends to shift toward the right half s-plane. System synthesis 
through use of the root locus technique amounts to proper placement of 
the closed loop poles and zeros. The process is by no means simple, but 
by use of some of the previously mentioned machinery, a large amount of 
the labor is circumvented. For a detail of design in terms of root loci, see 
Chap. 23 and Ref. 16. 

Relative Merits of Root Locus Method 

This method is theoretically exact and places in evidence the salient 
features of a closed loop system. Drawing the locus may involve a slight 
amount of work, but excessive labor is circumvented by use of mechanical 
aids. 

Major advantages of this method are: 

a. The behavior pattern of the entire closed loop can be shown in one 
simple diagram. 

b. Modes of the transient response are placed directly in evidence. 

c. Effects of variations in system parameters are placed directly in 
evidence. 

This is a relatively new method and is gaining widespread use. 

7. MISCELLANEOUS STABILITY CRITERIA 

There are many, many methods to perform stability analysis of linear 
systems. The following is a brief account of some methods not discussed 
in previous sections. For the interested reader, the references can be con- 
sulted for theory and details of operation. 

Hurwitz Criterion (Refs. 17, 18). This criterion is similar in nature to 
the Routh criterion and involves use of determinants. It is in general 
more laborious than the Routh criterion and offers information only with 
regard to whether or not all roots of the characteristic equation lie in the 
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left half s-plane. This method has been used to advantage in deriving 
other stability criteria such as stability boundary diagrams. 

Dzung's Criterion (Refs. 19, 20). This stability criterion is very 
similar to the Nyquist criterion except that it avoids the necessity of de- 
termining the location of poles of G(s)H(s) on the ^co-axis. It offers par- 
ticular advantage when G(s)H(s) is not known in factored form and Routh's 
criterion indicates poles of G(s)H(s) on the ^'co-axis. 

Wall's Criterion (Ref. 21). This stability criterion is similar to the 
Routh criterion and in many cases the computations are somewhat simpler. 

Stability Boundary Theory (Refs. 22, 23, 24). This method is nice 
in that some simple arithmetic calculations are made by using the coef- 
ficients of the characteristic function, the results are plotted on given 
charts, and stability is ascertained by inspection. The main disadvantage 
lies in the large number of charts required for higher order systems. 

Stability Plus Assurance of Margin of Stability (Refs. 25, 26, 27) 

By substitution, s' = (s + a) and/or s' = se jd in the characteristic 
equation, which corresponds to shift and/or rotation of the axes in the 
s-plane, and subsequent analysis of the resulting equation, stability plus 
assurance of a margin of stability can be ascertained. The substitution 
may result in an equation with complex coefficients. Analysis may be 
carried out by use of any of the following. 

Nyquist and Dzung Criteria. These criteria are general in nature and 
are applicable. 

Analog of Hurwitz (Refs. 28, 29), Wall (Ref. 4), Routh Criteria 
(Ref. 4). These criteria are similar to the criteria as described previously. 

Leonhard's Criterion (Ref. 30). This stability criterion is similar to 
the Nyquist criterion. 

Analog Computer Approach (Ref. 33). By simulating the equations 
which describe the physical equipment's behavior, it is possible to study 
system stability and performance characteristics. An entire part in Vol. 
2 is devoted to analog computers. 

8. CLOSED LOOP RESPONSE FROM OPEN LOOP RESPONSE 

As shown by eq. (1), the closed loop response of the general negative 
feedback system is a function of the forward and feedback transfer func- 
tions. Block diagram manipulation of the diagram in Fig. 1 leads to 
that shown in Fig. 97. That portion of the system shown in the dashed 
rectangle is a unity feedback system whose closed loop response is given 

by 

(71) d(s) _ G(s)H(s) 

R(s) 1 + G(s)H(s) ' 
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For any value of G(s)H(s), the closed loop response can be considered a 
vector given by 

(72) -^ = Me 3a . 

R{s) 

M is the magnitude of the vector where a is its direction in radians. 
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Fig. 97. General negative feedback system. 

Contours of Constant M and a. It can be shown (Ref. 31) that for 
unity feedback systems, certain curves in the complex plane correspond 
to loci of constant M and constant a. 

For the direct polar plot of G(s)H(s), the M loci are circles with 



Radius = 



M 



M 2 - 1 



and center at — 



M 2 



M 2 - 1 



on the axis of reals. 

The curves are shown in Fig. 98. The a loci correspond to circles passing 
through the origin and the — 1 point and centers at 

1 1 

hi 

2 2 tan a 

These a curves are shown in Fig. 99. These curves of constant M 
and a are useful for many purposes. An important use is that of obtaining 
the closed loop response from a plot of the open loop response, G(s)H(s). 
G(s)H(s) is superimposed on curves of constant M and constant a, and 
the closed loop magnitude and phase angles are obtained by inspection of 
the respective circles at points of intersection with G(s)H(s). 

For the inverse polar plot of G(s)H(s) it can be shown (Ref. 31) that 
the contours of constant M are circles with center at the —1 point and 
radius equal to 1/M. See Fig. 100. The contours of constant a are 
straight lines passing through the — 1 point with slope equal to a. 

The M and a contours in the inverse G(s)H(s) -plane plot are somewhat 
easier to use because of ease of construction. 



21-74 



FEEDBACK CONTROL 




-4 -3 -2-1 12 

Fig. 98. Circles of constant M in the (r(s)#(s)-plane. 
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-4-3-2-1 1 2 3 

Fig. 99. Circles of constant a in the (?(s)i/(s)-plane. 
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It is of interest to note that the M and a contours are perfectly general 
curves for unity feedback systems. In other words, G(s)H(s) is not re- 
stricted to those values of s on the ju axis. 




Fig. 100. Contours of constant M and a in the inverse G(s)H (s)-plane. 



Nichols Charts 

The information contained in the M and a circles, when plotted in 
terms of decibels and phase angle as shown in Fig. 101, are commonly 
referred to as Nichols charts because of the fundamental work first done 
by N. B. Nichols (Ref. 32). The curves shown in the figure have a mirror 
image about the —180° ordinate. The total curves correspond to the 
principal value of the logarithm given by eq. (28). Since the logarithm 
of a complex number is multivalued, eq. (27) the curves repeat as shown 
in Fig. 102. 

Stability Analysis on the Nichols Chart. Note that the —1 point 
in the G(s)H(s) -plane corresponds to the 0-db, —180° point in Fig. 101. 
For well-behaved, minimum phase G(s)H(s), the system is stable if 
G(ju)H(ju>) crosses the 0-db line to the right of the — 1 point on the Nichols 
chart. Figure 103 shows a stable system with a phase margin of +48° 
at gain crossover and a gain margin of 6.8 db at phase crossover. 

Exact Closed Loop Response. The procedure to obtain closed loop 
response from open loop response is as follows: 

a. Manipulation of the general negative feedback system to the form 
shown in Fig. 97. 
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Fig. 101. Nichols chart. 
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Fig. 102. Multiple Nichols charts. 
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Fig. 103. G(s)H(s) plotted on Nichols chart. 
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b. Plot G(s)H(s) directly or inversely with the corresponding M and 
a loci. G(s)H(s) on the Nichols chart is shown in Fig. 103. 

c. Obtain the closed loop M and a at points of intersection of G(s)H(s) 
with the M and a loci. 

d. Modify this response by l/H(s) to obtain the overall closed loop 
response. 

Approximate Closed Loop Response. The approximate closed loop 
response can be obtained by plotting the Bode diagram for G(s) and 



40 



20 



-20 



-90 



» -120 



Q -150 



-180 





^ 


































^s 


^ 






































dC( Sj 
R( Sj 


">\ 


^ 




1( S 


)| 


t\" 


1 




















y 








y~ 


iv 
































\ 


V \ C(S) \ 

\R(s)\ ' 




















y?\ 


iase angle o 


C(s) 
-R(s) 


(ap 


Dro 


x.) 

























































































































0.1 0.2 0.4 0.7 1 2 4 7 10 20 

w — >- 

Fig. 104. Approximate closed loop response. 
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[H(s)] * on the same sheet. The closed loop response is approximately 
equal to the lower of the two curves at any given frequency (see Fig. 104). 



(73) 



COO 



G(s) 



R(s) 1 + G(s)H(s) [1/0(8)] + H(s) 
when H(s) is much smaller than G(s) and G(s)H(s) is greater than 1. 

1 



and 



G(s) 



+ H(s) « H(8) 



C(s) 



R(s) H(s) 
when G(s) is much smaller than H(s) and G(s)H(s) is less than 1. 
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1 1 

+ #(*) 



G(s) G(s) 

and 

C(s) 

— « G(s). 

R(s) 

The approach is to approximate the closed loop response by the lowest 
portions of G(s) and [i7(s)] -1 . Assume all the breaks are simple and of 
multiplicity one or more. The phase diagram is drawn assuming the 
simple breaks of a minimum phase nature. 

The approximation is worst in the region of co where G(s) = [i7(s)] _1 . 
If necessary, the exact closed loop response can be obtained in this region 
by using the preceding exact method and the Nichols charts. 

Example. 

10 0.2s 2 

G(s) = H(s) = 

s(0.5s +1) (0.33s + 1) 

In Fig. 104 are plotted G(s) and [#(s)] _1 . The approximate closed loop 
response is shown as the heavy line and is approximated by the equation 

C(s) 10(0.33s + 1) 

(74) 



R(s) s(2s+ 1) (0.09s + 1) 



The phase angle curve is that corresponding to eq. (74). 

Note. There are many ways to investigate stability of linear closed 
loop systems. If used properly, they should all obtain the same result. 
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1. INTRODUCTION 

The frequency response technique of analyzing servo systems is used to 
facilitate both the analysis and synthesis operations (Chaps. 20 and 21). 
Often it is desirable to transform the results of the frequency response 
analysis into transient response form in order to interpret them more 
readily. Conversely, it is often desirable to transform the transient 
response performance requirements into frequency response form for 
synthesis purposes. These operations can be performed exactly by rigorous 
mathematical techniques; however, the operations are time consuming 
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and tedious, so it is often profitable to use less accurate but more easily applied 
techniques. The purpose of this section is to present some of the more 
useful techniques for relating the transient response to the frequency 
response and the inverse relations between frequency and transient 
response. 

2. RESPONSE CHARACTERISTICS DEFINED 

Transient Response. System response is often specified and inter- 
preted in terms of the characteristics of the transient response to a step 




1.000 



0.707 




Fig. 1. (a) Representative system response to unit step input. 
system frequency response (closed loop). 



(b) Representative 



input. The parameters which are most often used to describe the transient 
response are: 

C/R\ p , the peak value of the transient including any overshoot; 

t p , the time to the first peak if the response is underdamped and thus 
has an overshoot; 
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t s , the settling time, measured from the initiation of the step input to 
the time at which the system output no longer deviates more than a 
certain percentage, often 5 or 2 per cent from its final value; 

N, the number of oscillations it takes the system to "settle" or reach t s . 

Other parameters sometimes used to describe the transient response are : 

td, the delay time, measured from the initiation of the step input to the 
time at which the response has reached half the final value; 

t r , the rise time, which is the difference between the time at which the 
response has reached 10 per cent of the final value and the time at which 
90 per cent of the final value is reached. Rise time is also sometimes defined 
as the time from 5 to 95 per cent, and also as the reciprocal of the slope 
at the instant the response is 50 per cent of the final value. 

Figure la illustrates the definitions of these transient response param- 
eters. 

Frequency Response. System response is also often described in 
terms of certain frequency response characteristics. Chief of these are: 

M m , the maximum amplitude ratio of the closed-loop frequency re- 
sponse, which is sometimes designated C/R\ m ; 

co m , the frequency at which M m occurs ; 

ub, the bandpass frequency which is generally defined as the frequency 
at which the closed loop response is down 3 db from the nominal steady- 
state gain value. Figure 16 illustrates the definitions of these terms. 

3. RELATION BETWEEN TRANSIENT RESPONSE AND LOCATION OF ROOTS 
OF CHARACTERISTIC EQUATION 

Mathematical Relation. The open loop frequency response may be 
represented by the open loop response function in terms of its poles and 
zeros, roots of the denominator and numerator respectively, of the for- 
ward and feedback portions of the control system, 



(1) G(s)H(s) = 



NMN2J8) 

D 1 (s)D 2 (s) 
K(s + z n )(s + z l2 ) • • • (s + z 2l )(s + 222) 



s m (s ■+■ pn)(s + p 12 ) •••(« + p 2 i)(s + P22) 
The closed loop frequency response function can be obtained as follows: 
C(8) 0(s) >i(s)/Di(s) 



(2) 



R(s) 1 + G(s)H(s) 1 + N l (s)N 2 (8)/D l (s)D 2 (s) 
N 1 (s)D 2 (s) 
" ZMs)Z) 2 (s) + tfi(*)tf 2 00' 
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The closed loop poles can be found by factoring [Di(s)D 2 (s) -f Ni(s)N 2 (s)]. 
Substituting the proper Laplace transform for R(s), C(s) may be rep- 
resented by a sum of terms such as 

At A 2 A 3 

(3) C(s) = — + — — + — — + • • • , 

s s + a 2 s + a 3 

where Z\ n are the roots of N\, z 2n are the roots of N 2 , p\ n are the roots of 
D\, p 2n are the roots of D 2 , and where the constants A\, A 2 , A 3f etc., are 
found by partial fraction expansion. 

The time response function is then found by performing the inverse 
Laplace transformation to get 

(4) c(t) = Ai + A 2 exp (-a 2 t) + A 3 exp (-a 3 t) + • • •. 

In general, this function must be plotted to determine such parameters 
as peak overshoot and settling time which are often of prime importance. 
The straight mathematical approach is impractical for any but simple 
systems because of the amount of tedious work involved and the fact that 
it does not lend itself to system synthesis. 

Approximate Approach 

The time response can be estimated quite accurately by noting the 
location of certain predominate closed loop poles and zeros in the complex 
frequency plane (s-plane). The closed loop pole-zero configuration may 
consist of one or more pairs of complex poles and several real axis poles 
and zeros, and perhaps complex zeros. Ordinarily, one pair of complex 
poles will be of primary importance because of its frequency or damp- 
ing ratio. For example, if a system contains two pairs of complex poles 
which have natural frequencies that differ by as much as 10 to 1, the 
designer may ordinarily consider either pair as dominant and perform an 
analysis in two parts, considering first one pair and then the other. 

For many cases it is reasonably accurate to neglect all but one complex pole 
pair and to consider the transient response to be made up of the dominant 
pair of complex poles and various groupings of real axis poles and zeros. 
This assumption is made in the following discussion. 

Only underdamped systems are to be considered here. Overdamped 
systems may generally be analyzed quite easily by the normal mathe- 
matical techniques. 

Dominant Pair of Complex Poles. To determine the effect of the 
s-plane pole-zero configuration in the system transient response, it is 
convenient to first consider the relation between a single pair of complex 
poles on the s-plane and the characterizing parameters of the time response. 



TRANSIENT AND FREQUENCY RESPONSE 



22-05 



The additional effect of the real axis poles and zeros will be considered 
later. 

The expression for the closed loop frequency response function contain- 
ing one pair of complex roots is 



(5) 



G(s) 



W 



C(s) = 

R(s) 1 + G(s) s 2 + 2fa s + coo 2 



(s + o-q + ji» d ) (s + o-o - j<a d ) 



where co = natural frequency, 
f = damping ratio, 
0"o = fto = damping exponent, 

wj = u v 1 - f 2 = natural damped frequency, or oscillation fre- 
quency. 

The parameters co , f , o- and ud are shown on the s-plane in Fig. 2. 



Complex 
pole 




Fig. 2. One pair of complex roots and significant related parameters, f = cos t/a. 



Figure 3 illustrates that for constant values of natural frequency, co , the 
complex roots or poles of eq. (5) generate circles on the s-plane as the 
damping ratio f is varied. Radial lines from the origin are generated by 
holding f constant and varying coo- 

Figure 4 illustrates that holding a , the exponential damping factor, 
constant forms lines parallel to the imaginary ( t /co)-axis on the s-plane. 
Similarly, maintaining constant values for the damped frequency, co^, forms 
lines parallel to the real (a) axis. 
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The expression for the time response to a unit step input is 
(6) c(t) = 1 + (coo/cod) exp (— o-oO sin {w d t — <M, 

where fa = arctan co^/ — o- 



= arctan Vl - f 2 /f. 
Figure 3 also illustrates that constant values of \pi correspond to constant 



^ 2 = cos _1 f, 



s-plane 



i^i = cos -1 f x 




Fig. 3. Constant wo is a circle; constant f is a radial line. 





s-plane 
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Fig. 4. Illustrating lines of constant u>a and <tq. 

values of £*. From eq. (6) the characterizing parameters of the transient 
response can be derived. The equations for the more important ones are : 

(7) t p — ir/oia — 7r/cooVl — f 2 , time required to reach first peak. 

(8) t s = 3/cr = 3/fcoo, time required to settle to within 5% of final 

value (=4/fcu for 2%). 
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(9) C/R\ p = 1 + exp (— irf/Vl — f), the peak value of the ratio of 

= i + exp (_ Wo / ttd ) output to input. 

(10) iV = ts/i^ir/ud), number of oscillations to settle to within 5% 

of final value. 

= 3V 1 - r 2 /27rr 

Equations (7) through (10) relate the position of the dominant pair of 
complex poles on the s-plane to certain transient response parameters. 
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Fig. 5. C/R \ p , N, uot P , wo<* versus f for system composed of two complex poles. 
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The relationships between these parameters and the damping ratio, f, 
are plotted in Fig. 5. 

Similarly certain closed loop frequency response parameters can be 
related to the position of the poles. 



(11) M m = l/(2fVl - f ), < f < 0.707, maximum frequency re- 
= 10 707 < f < 1 sponse ratio of output to 

input (also defined as M m ) . 

2.0 



1.6 



1.4 



1.2 



1.0 



0.8 



0.6 



0.4 



0.2 



















































^■M m 










-a> 6 /aj 








M c -K 



































0.2 



0.4 0.6 0.8 

Damping ratio, f 



1.0 



Fig. 6. M m , M c , »&/wo, versus f for system composed of two complex poles. 
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(12) M c = l/(2f), response ratio at the frequency corresponding to the 
natural frequency, or corner frequency. 



(13) co b = w V 1 ~ 2 ^ 2 + ^ 2 - 4 f 2 + 4 ^ 4 > bandpass frequency, at 

which response ratio is 
0.707. 

Figure 6 shows graphically the relationship between these parameters and 
damping ratio. 

Effect of Real Axis Poles and Zeros. The mathematical expression 
for a system whose dynamic characteristics can be described by one pair 
of complex roots, one real pole, and one real zero is 



(14) 



w 2 p(s + z) 



C{s) 



R(s) z(s + p)(s 2 + 2fa> s + coo 2 ) 



The expression for the transient response to a unit step input may be 
written as 



"o 2 (z ~ V) 



(15) c(t) = 1 ^f- exp (-p0 



(zp d \ / V \ / w o\ 
— ) [-=) ( — ) exp (-<r t) sin (<a d t - fa + fa — ^ 
Z / \pPd/ \«d/ 



-fa) 



<PPd 

where ppa — distance from p to p<i, 
zpa = distance from z to pj. 
A graphical representation of this system is contained in Fig. 7. 




Fig. 7. Illustrating a system with one pair of complex poles, one real pole, and one real 

zero. 
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The term [w 2 (z — p)/z(ppd) 2 ] exp (— pt) in eq. (15) is neglected in 
determining the following expressions for the characteristic parameters 
of the transient response. This approximation is valid when p is much 
larger than cr (at least 3 times as large). The expressions for C/R\ p , t p , 
and t s are: 



(16) 

(17) 
(18) 



\ Z / \VPd/ 



exp 



— O- (7T — 1^3 + ^4) 



VPdf l m 

= (t — ^3 + ^4)/«di 

^s = 3/fw . 



The settling time, t 8 , remains the same as before since the exponential 
pole term in eq. (15) has been neglected. Thus, N also remains unchanged. 
For multiple poles and zeros the expressions for C/R\ p and t p in eqs. (16) 
and (17) become: 



(i9) ||.( n ^)(n^)exp 

Rip \ Q VqPd/ \ q Zq / 



— <Tq{tT ~ 2^3 + 2^ 4 ) 



«d 



(20) 
where 2^ 3 



tr 



IT — 2^3 + 2^4 



^d 



sum of all angles between the real axis zeros, and the domi- 
nant pole, 



2^4 = sum of all angles between the real axis poles, and the domi- 
nant pole, 

JJ = product of the ratios of the poles to the distances from the 

q PqPd poles to complex pole at point pa, 

tt z oPd 

ll = product of the ratios of the distances of the zeros to point pd, 

« Zq to the zeros. 

As noted before, eqs. (19) and (20) are approximate, based on the assump- 
tion that p is large compared to cro, which is realistic for many practical 
systems. Conclusions reached from a study of eqs. (19) and (20) are: 

.1. The time to peak, t p , is inversely proportional to o>d, the damped 
natural frequency. 

2. The addition of a pole increases t p and decreases C/R \ p , the magni- 
tude of the peak. 

3. The addition of a zero decreases t p and increases C/R \ p . 

4. If a pole and zero are close together (dipole), their net effect on t p 
and C/R \ p is negligible. 
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5. Poles and zeros far out on the real axis have little effect on t p and 
C/R\ p . 

If the assumption of the real poles and zeros being large compared with 
the damping factor (p ^> a ) is not valid, the values of C/R \ p and t p can 
still be estimated though not by the simple use of eqs. (19) and (20). 

f-l - B sin (u d t + if/) 

■ 1 -Ae~ pt - B sin (u d t + \p) 




Fig. 8. Illustrating the effect of a significant real pole on C/R \ p approximations. 

The magnitude of the coefficients of the exponential terms such as the one 
in eq. (15) can be calculated, as outlined in the next section, and then the 
effect of these terms on C/R \ p and t p can be estimated. By referring to 
Fig. 8 as an example, it is apparent that the value of the simple exponential 
term at time, t p , must be subtracted from the approximate curve to give 
a more exact value of C/R\ p . In other cases this correction might have 
to be added. Of course this process becomes more difficult as the number 
of significant poles and zeros increases. 




Time 
Fig. 9. Response with a dominant real pole and a pair of complex poles. 

The addition of poles and zeros to the closed loop response function of a 
control system may result in a response function whose dominant charac- 
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teristic is that of the real pole rather than the complex pair. For such a 
case the system response to a unit step input might be as illustrated in 
Fig. 9. 

Coefficients of Transient Response Terms 

Frequently, it is desired, as soon as the closed loop poles are found by 
graphical or other means, to determine the exact expression for the time 
response. This may be especially true when it is obvious that two pairs 
of complex poles are significant, i.e., they are both located about the same 
distance from the origin. 

The coefficients of the terms in the equation describing the transient 
response of the system may be calculated by a formula developed in Laplace 
transform theory. This formula may be interpreted in terms of the pole- 
zero configuration of the root locus plot for the system. 

To illustrate this a system described by the following equation is assumed. 

C(s) K(s + z) 

(21) K 



R(*) (* + Pi)[(« + *o) 2 + "/] 



If the appropriate Laplace transform for R(s) is substituted into eq. (21), 
the expression for C(s) can be written. Assuming a unit step input in 
this case (R (s) = 1/s), then 

K(s + z) 

(22) C(s) = — -• 

s(s + Pl )[(s + ctq) 2 + co d 2 ] 

In terms of partial fractions eq. (22) can be written as (see Chap. 20) 



(23) C(s) = K 



A Ai A 2 A 3 

_l 1 — — — _j_ 



s s + px s + (o- + jm) s + (o- - ju d ). 
The transient response equation for this system is 

(24) c(t) = K[A + A x exp {- Vl t) 

+ A 2 exp (<r -J- ju d )t + A 3 exp (o- - ju d )t]. 

Determining the Coefficients. The formulas for the coefficients A , 
A 1} and A 3 are: 

(25) A Q = sC(s)< 

(26) . A 1 = (8 + Pl )C(8)\ 



s=0 Pl(o"0 2 + 0}/) 

z ~ Pi 



-PlK'o - Pi) 2 + oo d 2 ] 
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(27) A 2 = [s + (<r + fad)]C(s)\ 
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l 8=— (iro+joid) 

Z — (o-Q + i"d) 



— (cr + fad)[Pi — (o-o + fad)][(<ro — i«d) - (<r + j&} d )] 
2 — (o-Q + Mi) 



2jco d (<r + fa d )(pi — (To — fad) 
(28) A 3 = [s+ (cr -fa d )]C(s)\ 

<s=—(oo—jb>d) 
z — (o"o — JWd) 



— (o-o — jwd)foi — (o-o — iwd)][(o"o + iwd) - (<r - i"d)] 

— 2fa d (<TQ — fad)(Pl — O"0 + JW<*) 

Note that these coefficients are the ratios of vectors in the root locus 
plot. For example, consider Fig. 10 which illustrates the pole-zero con- 

+ju 
J* 




Fig. 10. Vectors for determining A\ coefficient. 

figuration of the system under consideration. With this plot the value of 
any coefficient can be determined by drawing vectors from all other poles 
and zeros to the pole or root which corresponds to the coefficient. The 
coefficient is then the ratio of vectors from the zeros to those from the 
poles. Figure 10 shows the vectors for the calculation of A x . From this 

-Pi + z 



(29) 



A l = 



■Pi[(-Pi + o- ) +jo)d][(-pi + o- ) - ja d ] 
z - Pi 



-Pi[(o"o - Pi) 2 + "d 2 ] 
which agrees with eq. (26). 
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Similarly, the other coefficients may be determined from the root locus 
plot. Figure 11 shows the same system with the vectors oriented for de- 
termination of the A s coefficient for one of the complex roots. From this 



(30) 



Ao = 



A 3 



1 A3 1 /fa 



X1A2A4 (MAh)(M/fa)(MAh) 
|x 3 1 



i|Xi| |x 2 | |x 4 | 

since fa = -\-j or +90°. 



I fa — fa — fa , 













+J03 








^> 




x 2 \ 








^yf 4 




\fa 


"N^ 2 \ 


. \ 


c 


-Pi 




—2 




— -c — 







Fig. 11. Vectors for determining A3 coefficient. 



In like manner the A 9 coefficient can be determined as 



(31) 



A» = 



|Xa| /-fa 



(|Xi| /-fa )(M /-fa) (M /-fa) 



IX3I 



-i|Ai 



x 4 



/—{fa - fa — fa) 



The Xi, X 2 , X 3 , and X 4 vectors can be expressed in terms of the pole-zero 
locations in the root locus plot. When this is done eqs. (25) to (28) are 
the results. These coefficients can be evaluated conveniently by use of 
the Spirule, although a ruler and protractor will suffice (Ref. 1). 

The time response of a system such as the one being considered here is 
usually expressed in equation form with a sine or cosine term instead of 
the complex exponents. This is illustrated in the following equations. 

(32) c(t) = A Q -f A x exp (-pi<) + A 2 exp (— a - j<o d )t 

+ A 3 exp ( — 0-0 -f- jm)t. 
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From eqs. (30) and (31) the coefficients A 2 and A 3 may be expressed as 

(33) Az= ] -^(M, 

J 

(34) A 2 = -1^1/-A 3 . 

3 

Combining these equations with eq. (32) yields 

(35) c(t) = A -f At exp (-piO + exp ( — <r o X 

l^sl |A 3 | 

exp (—ju d t - j/A 3 ) -\ — exp (ju d t + j/A 3 ) 



I J 3 

= A + Ai exp (-piO + 2 |A 3 1 exp ( — <r t) sin (w d t + /A 3 ), 

where|A3l= |x 1 ||x 2 ||x 4 | 

and these vector lengths and angles are shown in Fig. 11. 

4. RELATION BETWEEN CLOSED LOOP AND OPEN LOOP ROOTS 

Mathematical Relationship. The closed loop frequency response 
function may be readily written in terms of the open loop 'function as 

C(s) G(s) 

(36) 



R(s) 1 + G(s)H(s) 

where G(s) = open loop transfer function of forward element 
H(s) = feedback element. 

If these are written as 

(37) G(s) = NM/Dda); H(s) = N 2 (s)/D 2 (s) 

then 

C(s) N^/D^s) tfiOOZM*) 

(3s) 



R(s) 1 + N 1 (s)N 2 (s) D 1 (s)D 2 (s) + Ni^N^s) 

2>i(8)D 2 (s) 

The closed loop poles are thus the roots of the denominator of eq. (38). 
This is generally a high order polynomial and constitutes a tedious task 
if it is to be factored. For this reason methods of estimating closed loop 
roots from open loop roots are useful. 
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Graphical Method of Determining Roots. A convenient way to 
obtain the closed loop poles {roots of [N~i(s) N 2 (s) + D x (s) D 2 (s)]} is to 
use the root locus technique of graphically plotting loci of the closed loop 
poles as functions of the open loop system gain (see Chap. 21). These 
loci may be plotted from the open loop pole-zero configurations on the 
complex frequency plane as shown in Chap. 21. The plotting is simplified 
considerably by use of the Spirule plotting tool. 

The selection of the open loop gain to give the proper system response 
may be determined either by use of frequency response or root locus 
synthesis techniques. In either case the configuration may be examined 
for its transient response characteristics. 

A method of performing the reverse operation, working from closed 
loop to open loop roots, is presented in Chap. 23. 

An Iterative Process of Determining Closed Loop Roots. The 
closed loop poles {roots of [iVi(s) N 2 (s) + Di(s) D 2 (s)]\ can be found 
more easily by mathematical techniques if known roots can be factored 
out and leave a simpler polynomial. A technique is available (Ref. 2) 
which allows closed loop poles to be found after the open loop response 
characteristics of the system, including the open loop gain, have been de- 
termined. 

The rules for determining closed loop poles by this method are the following : 

1. An open loop zero located at a frequency lower than the crossover 
frequency, co c , is approximately equal to a closed loop pole. 

2. An open loop pole located at a frequency higher than the crossover 
frequency, co c , is approximately equal to a closed loop pole. 

These rules are used to make the first approximations for the closed 
loop poles. These approximate values may be refined by iteration using 
the following expressions : 

For open loop zeros much smaller than oi c , 



(39) ^ + Zl y 



(s + z,) n 



H(s)G(s) 



where pi = closed loop pole, 

z\ = open loop zero much smaller than w c , 
Pi_i = value of pi found by previous iteration (equals z\ for first 
iteration), 
n = order of open loop zero. 

For open loop poles much larger than co c , 

(40) (pi + Pl ) n « -(« + p 1 ) n H(s)G(s) | s=Pi _ x , 

were Pi = closed loop pole, 
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Pi = open loop pole much larger than u c , 
Pi_i = value of pi found by previous iteration (equals pi for first 
iteration), 
n = order of open loop pole. 

If n is greater than unity, n values for pi will result with each iteration. 
Further refinement should continue only on those values of Pi remaining 
far from a> c . If the value of pi approaches co c the accuracy of the technique 
is poor. 

After these closed loop poles are found with sufficient accuracy by itera- 
tion, they may be factored from the closed loop polynomial characteristic 
equation for the system. The resulting lower order polynomial may then 
be more easily factored. 

In general if the open loop poles and zeros are larger or smaller than 
die by a ratio of 3 to 1 or greater, two iterations will result in sufficient 
accuracy for finding the first closed loop poles. 

The coefficients of these transient response terms may be calculated as 
indicated in the previous section. 

Example. Determining Roots. Assume the open loop transfer function 



(41) 



G(s)H(s) = 



400(s + 1) 
s(s + 2)(s + 10) J 



As previously stated: open loop zeros less than w c are approximate closed 
loop poles. The crossover frequency, co c , is 4 rad per second, as may be 
easily determined from a graphical plot of eq. (41). Therefore, 



(42) 

To refine this approximation 



p,-_i = -1.0. 



(43) 



(Vi + 1) « 



- S ( S + 2)(s+10)< 



400 



s=— 1 



l(-l + 2)(-l + 10) 



2n 



400 
~ 0.20. 

(44) Pi « -1 + 0.20 = -0.80. 
This may be repeated, 

(45) (Pi + 1) « - 

« 0.20. 



-0.8(-0.8 + 2)(-0.8 + 10)' 



400 



(46) 



Vi 



-0.80. 
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Similarly, open loop poles larger than co c are approximate closed loop 
poles, so 

(47) p t -=-10,-10, 

„ -400(s 4- 1) 

(48) (P;+10) 2 = 



s(s + 2) s= _io 
- + W = +40. 

(49) Pi « -3.7, -16.3. 

Since the co c is 4, only the larger root can be expected to be useful. With it, 
repeating the process gives 

-400(-16.3 + 1) 

(50) (p t - + 10) 2 = = 26.3, 

-16.3(-16.3 + 2) 

(51) Pi = -4.9, -15.1. 

Again, 

-400(-15.1 + 1) 

(52) ( Pi + 10) 2 = -. 

-15.1(-15.14-2) 

(53) pi = - 15.35. 

The two closed loop poles thus determined are s = —0.80, —15.35, and 
hence they may be factored out of the expression 

(54) JVi(s)iNT 2 (s) + Z) 1 (s)i) 2 (s) = 400(s + 1) + s(s 4- 2)(s + 10) 2 

= s 4 + 22s 3 + 140s 2 + 600s + 400 
to give the closed loop poles near co c as : 

(55) s 2 + 5.85s 4- 33.3 = (s - 2.93 + j'5.34)(s - 2.93 - i5.34). 
Thus, the closed loop poles are: 

(56) s = -0.80, -15.35, (-2.93 + .75.34), (-2.93 - i5.34). 

In this example if H(s) is other than unity these roots are not the roots of 
C(s)/R(s), but are the roots of H(s)[C(s)/R(s)]. 

5. DESIGN CHARTS RELATING OPEN LOOP FREQUENCY RESPONSE AND 
TRANSIENT RESPONSE 

An approximate method of relating steady-state frequency response 
characteristics and transient response characteristics has been described 
(Ref. 3). It makes use of a series of charts which indicate the type of open 
loop attenuation curves required to produce desired closed-loop responses. 
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If a servo system falls within the group considered in the charts, this 
method enables the designer to take a set of specifications setting forth 
steady-state frequency and/or transient response requirements and quickly 
estimate the necessary open loop characteristics. The charts also permit 
the designer to estimate the effect of changing various system parameters 
to give him a better understanding of the system. 

Description of Charts. The symbols used on the charts are defined 
below and illustrated in Fig. 12. 



C 
R 
C 
R 

Oir. 



maximum ratio of the closed loop frequency response (M m ) 

peak value of the ratio of controlled variable to input for a step 
function input 

ratio of the frequency w m at which C/R \ m occurs to the frequency 
a) c at which the straight-line approximation of the open loop re- 
sponse is db 

ratio of a><, the lowest frequency of oscillation for a step input, to 
co c , the frequency at which the straight-line approximation of the 
open loop response is db 

o) c t p the frequency, co c , at which the straight-line approximation of the 
open loop response is db times the response time t p measured 
from the start of the step function until C/R \ p occurs 

co c £, the frequency, co c , at which the straight-line approximation of the 
open loop response is db times the settling time, t s , from the 
start of the step function until the output continues to differ from 
the input by less than 5 per cent 

Indicated in Figs. 12a, b, and c are these various characteristics in terms 
of the familiar curves of the open loop transfer function, the frequency 
response, and the transient response to a step input. The charts, Figs. 
13 to 30, were prepared for a system with an initial open loop attenuation, 
Fig. 12a, of 20 db per decade. However, the shape of the curve near 
db is of greatest importance, so the curves may also be used for systems 
with initial attenuation slopes of to 40 db per decade. 

Limitations. Of necessity the charts may be used for the analysis and 
synthesis of a somewhat restricted class of servomechanisms. Their use 
is restricted to : 

(a) Linear systems, or those which may be considered linear for a re- 
stricted range of operation. 
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=5 Ml 




Frequency, radians/second 
(a) 



"3 



40 db/decade 




Steady -state 
frequency 
response 



Frequency, radians/second 
(b) 



Transient response 

following a unit 
step function input 




Time, seconds 
(c) 

Fig. 12. Sketches showing nomenclature used in the design charts of Figs. 13 to 30. 

(b) Single loop, unity feedback systems containing only series elements ; 
of course a multiloop system may be considered if the inner loops are 
reduced to equivalent series elements. 

(c) Systems whose open loop characteristics fall into the category of 
servomechanisms described by Fig. 12. However, systems which os- 
tensibly are not of this type may often be approximated by some which 
are, especially if the required approximations occur at an appreciable 
distance from the crossover frequency, co c . 

(d) A step function as the form of the input signal producing the tran- 
sient response. 
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Fig. 13. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 



22-22 



FEEDBACK CONTROL 



2.4 

2.2 

2.0 

oj 1.8 
O 

1.6 
1.4 



1.2 



1.0 



































oil t° 

6)3 to 


r 
w 2 = 


1 1 1 1 1 
= 40 db/decade 


































00 = 40 db/ decade 
w 3 




































w c ~ 

c\ - 






































































CI 




















































1 
/ 














































/ 
/ 




/ 
1 












I 




1 

i 


























1 




1 
1 










l 






I 

l 




1 

• 




















1 






1 / 
I / 










1 
1 






I 

I 


/ 


























/ 


' / 








i 
I 








1 / 

i / 


/ 




















/ 

f / 






t 








i 


1 
j 








1 / 
i/ 


/ / 


















/ 








y 








i. 










y 


















/ 










^y 




s*^. 


4 






y(fl 


= i^sy 
















Mi = 80 
.... i i 








Mi = 60 


Mi ="40 




^ 


y 








-*^'v.\ = 20 















0.01 



on 

03c 



0.1 



1.4 
































«i to 
CO3 to 


.02 = 


1 1 1 1 1 

: 40 db/decade 
































00 = 40 db/decade 

"3 . 


1.2 



































Uc 
















/ 




1 














Oit 








3-1.0 














<• 




j 
















1 03m 
03c 

03c tp — — 

03 c ts 


















J I 












//^^ 


^y 1 


0.8 




-^ 








'l 


/ 




4 i 










1 

t 


1^/1 


1 
1 






\ 




/ 


1 




^'\ / 


1 i 










t 


/ i 


1 
/ 


g 
2 0.6 






1 


/ 




i 




V 


j 


/ 




/ 
V 




4 


/ i / 




/ 
















/ 
f 


V 


.' 






/ 1 






/ 


\ 


•\ 


'- 


/ 




f 












c 
CD 

g-0.4 




/ 


■ 










/ 
/ 




/ 


' 


/ 


/ 


/ 


/ 


^ 
































/ 




/ 


/ 




/— 
















0.2 


















~- 












^■r- 


- 
















Mi = 80 








Mi = 60 




Mi = 40 mi = 30| Ml = 20 














n 















































0.01 



0.1 



1 



Fig. 14. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 15. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 16. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 17. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 18. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 19. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 20. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 



TRANSIENT AND FREQUENCY RESPONSE 



22-29 



2.4 V- 

2.2 
2.0 



a; 1.8 



1.6 



1.4 



1.2 



1.0 



Wl 


1 
to 0> 


— 


1 1 1 1 
60 db/decade 






1 
1 
















1 














C03 to oo = 40 db/decade 
w 3 _ 

zr c ~* 

C\ 
R\m 




1 








1 








1 


















/ 
1 








1 
1 


























1 

1 








1 

1 






J 
1 
















C\ 










1 








1 






/ 
/ 
























/ 








1 








/ 
/ 


































/ 








1 

1 








/ 


































/ 
/ 








1 








/ 


































/ 
/ 






/ 










1 

1 


/ 
































/ 
















1 

1 


/ 

, 1 
































' > 




1 
1 












1 / 

1 / 


1 
1 




1 


























/ 
/ 




/ 


/ 
















, 
































/ 
/ S 










/ 


' 




































•, 








s 


> 






// 


/ 
























/■ 


l l 




- 80 Ml = 60 Mi = 

I 1 


41) - 
Ml 


= 30 








^-^^^i = , 20 













0.01 



0.1 




Fig. 21. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 22. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 23. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 24. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 25. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 26. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 27. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 28. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 29. Charts giving comparison of steady-state frequency response and transient 
response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). 
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Fig. 30. Charts giving comparison of steady-state frequency response and transient 

response following a step. See Fig. 12 for definition of nomenclature (Ref. 3). Note 

that W2/co c is the abscissa for these charts. 
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Uses of Charts. Figures 13 through 30 are comparisons of steady-state 
frequency response characteristics and transient response following a step 
function of input as a function of coi/co c (Ref. 3). The information pre- 
sented in Figs. 13 to 30 is useful for analysis, that is, determining the 
response of systems already designed, or for synthesis, that is, determining 
what sort of system will be required to do a specified job. Typical examples 
of each are presented in the following. 

Example 1. Analysis. Determine, approximately, the value of C/R | m 
and the frequency at which it occurs, and the magnitude of the peak over- 
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Fig. 31. Gain plot for example problem. 



100 



shoot to a step function input and the time when it occurs for a system 
having the open loop transfer function, 

(57) C/E = 10/s(l + 0.1s), 
which is drawn in Fig. 31. From this, 

«3 = w c = 10, 

(58) coi = o> 2 = 1.0, 

m = 20 db. 

The values for «i and hi were arbitrarily selected; of course, choosing a 
value for one fixes the value of the other. Since there is no segment be- 
tween coi and co 2 with either 40 or 60 db per decade slope the chart with 
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the lower attenuation rate 40 will be used. By entering the chart in Fig. 
13 with the following parameters : 

m = 20, wi to co 2 = 40 db/decade, 

(59) coi/co c = 0.1, co 3 to oo = 40 db/decade, 
W3/o) c = 1.0, 

the desired information may be obtained. 

C/R\ m « 1.16, C/R\ p « 1.18, 

(60) COm/oic » 0.7, — « 0.36, 

u m « (0.7) (10) = 7.0, t p ~ 0.36 sec. 

Example 2. Synthesis. The requirements of a position control system 
are assumed to be set forth in the following set of specifications : 

1. C/R\ m = 1.3 or less. 

2. co m = 2 cycles per second or more. 

3. Velocity error coefficient (K v ) is 200 sec -1 . 

4. The attenuation rate of the open loop control will be 60 db per decade 
for frequencies greater than 120 rad per second. 

The problem is to determine the open loop transfer function of a suitable 
control for this application. 

The frequency, co c , must be considerably greater than a m , so as a first 
assumption assume that w c = 30 rad per second, and that co 3 = 120 
rad per second. For the specifications given there are many solutions to 
the problem. Figure 28, for which cos/co c = 4, shows that for nx = 40 db 
and cox/coc = 2/30 = 0.067. 

C/R\ m = 1.25, 

(61) o) m /co c = 0.6, 

<a m = (0.6) (30) = 18 rad/sec. 
Thus, an open loop transfer function satisfying the specifications is 
C(s) 200(1 + 0.2s) 2 



(62) 



E(s) s(l + 0.5s) 2 (l + 0.00833s) s 



Synthesis by means of the charts is basically a trial-and-error process- 
assuming the solution and checking it. 
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Table 1. Rule-of-Thumb Approximations 



Parameter 


Approximation 


Time to 


l p ~ Tr/o} c 


peak 


where t p = time from step in- 




put to peak value 




of response tran- 




sient, seconds 




o) c — open loop crossover 




frequency, radians/ 




second 



Peak over- 
shoot 



C/R\ p * 
where C/R \ p 



M„ 



* 0.85ilf m 

= peak value of 
transient re- 
sponse to a 
step input 

= maximum val- 
ue of closed 
loop frequency 
response 



Damping 
ratio 



f = 1/(2M C ) 
where f = damping ratio 

M c = value of closed loop 
frequency response 
at the corner fre- 
quency 



Settling 
time 






3Vi - r 2 /f«d 
5V1 - rvr^d 



Remarks 

In Chestnut and Mayer's charts 
it is evident that for this gen- 
eral class of servomechanisms, 
those with a dominant complex 
pair of closed loop poles, the 
open loop crossover frequency, 
oj c , times the time to peak, t p , 
is about 3 or r. In other words, 
the time to peak is about half 
the period corresponding to the 
open loop crossover frequency. 

The peak value of the transient 
response, C/R \ p , to a unit step 
input is generally less than the 
maximum steady-state value, 
M m , of the closed loop fre- 
quency response. The maxi- 
mum value of C/R \ p generally 
approaches 2.0 while the maxi- 
mum value of M m approach- 
es infinity. For many applica- 
tions "good" servos are those 
with the values of M m be- 
tween 1.3 and 1.5. For these 
servos M m is generally 10 to 
20% greater than C/R\ p . 

The damping ratio may be ap- 
proximated from the value of 
the closed loop frequency re- 
sponse of the system at the 
corner frequency, co c (the fre- 
quency at which the lines 
asymptotic to the log magni- 
tude curve intersect). This is 
exact for a second order system. 
Of course this relationship may 
also be used to estimate M c , 
knowing the damping ratio. 
In addition, M c is approxi- 
mately equal to M m for sys- 
tems with low damping ratios. 

The settling time, t s , is generally 
defined as the time for the sys- 
tem to settle to within 5 or 
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Table 1. Rule-of-Thumb Approximations (Continued) 



Parameter 

Settling 
time 
(continued) 



Equivalent 
time con- 
stant 



Oscillation 
frequency 



Approximation 

where t s = time for response to 
step input to settle 
to within some per 
cent of final value, 
seconds 
Teq = time for response to 
reach 63% of final 
value 
oia — damped natural 

frequency, radians/ 
second 
f = damping ratio 

^eq « 1/We 

where 7% — time for response 
to step input to 
reach 63% of final 
value, seconds 
a) c = gain crossover 
frequency, radi- 
ans/second 

&t ~ co m ~ 0.75co c 
where co< = oscillation fre- 
quency of transient 
response, radians/ 
second 

co TO = frequency at which 
M m occurs, radi- 
ans/second 

o) c = open loop gain 
crossover fre- 
quency, radians/ 
second 



Remarks 

sometimes 2% of the final val- 
ue. In either case it is quite 
difficult to predict t s for an 
underdamped system because 
it is subject to fluctuations of 
about one-half the period of 
oscillation for only small 
changes in system parameters. 
However, approximations (see 
eq. 18) can be made. The last 
approximation is for an over- 
damped system. 

This relationship is exact for a 
simple single time constant sys- 
tem, but also quite good for the 
general case (Ref. 4). 



The frequency of oscillation of the 
transient response, co t , is gener- 
ally about equal to the fre- 
quency, oo m , at which the fre- 
quency response peak, M m , 
occurs. Both co TO and io t are 
usually less than co c , the open 
loop crossover frequency. For 
the "good" servos with M m 
= 1.3 to 1.5 an approximate 
relationship is as indicated. In 
this approximation oo t is used to 
mean essentially the same thing 
as aid, the damped natural fre- 
quency, previously defined for 
a system with a dominant com- 
plex pair of poles. The use of 
co t places no restriction on the 
system characteristics; how- 
ever, generally, there is no sig- 
nificant difference between ia t 
and a)d. 
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Table 1. Rule-of-Thumb Approximations (Continued) 



Parameter 


Approximation 


Remarks 


Rise time 


trCO t ~ tr^m ~ 1-3 


The system's rise time, t r , which 




where t r = rise time (10 to 


is here considered to be the 




90%) 


time for the response to a step 




wt = (denned above) 


input to go from 10 to 90% of 




co m = (denned above) 


its final value may be approxi- 
mated as indicated for systems 
with a M m value of about 
1.3 to 1.5. 


Phase mar- 


7c ^ 40° 


A phase margin of 40° at the 


gin at 


where j c = open loop phase 


unity gain (crossover) frequency 


crossover 


margin at the 


generally corresponds to a 


frequency 


crossover fre- 


M m ratio of approximately 




quency 


1.5. Since this value of M m 



is the maximum ordinarily con- 
sidered feasible, the phase mar- 
gin should be 40° or greater. 

6. APPROXIMATE RELATIONS— RULES OF THUMB 

There are several approximations or rules of thumb which can be quite 
useful when time or facilities are not available for a more exact analysis. 
They may also be used as rough checks on the results of a more extensive 
analysis. The more common of these rules of thumb are presented in 
Table 1. They must be used with caution because, being approximations, 
they cannot apply with equal validity to all servo systems; and the approxi- 
mations for transient response are applicable only for step inputs. 

7. NUMERICAL AND GRAPHICAL TECHNIQUES OF RELATING TRANSIENT 
AND FREQUENCY RESPONSE 

The numerical techniques presented involve only routine calculations 
and provide a point by point determination of the related response without 
the need of obtaining the closed loop poles or other intermediate quantities. 

The methods presented require the following assumptions: 

(a) The system is linear. 

(b) The system frequency response approaches zero as the frequency 
approaches infinity. 

(c) The system's transient response begins with the system initially at 
rest. 

(d) The system is stable. 

These requirements are satisfied by most servo systems. Even a non- 
linear system may generally be considered linear over a restricted operating 
range. 
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Determining Transient Response from Frequency Response. A 

relatively simple method for obtaining the time response to an impulse 
function input, knowing the frequency response, was developed by Floyd 
(Ref. 5). He derives the exact inverse transformation and then presents a 
method for numerically performing the necessary integration. The exact 
transformation is 



(63) 



r) i 



c(t) = (2/tt) I {Re [G(ju)] cos tw} dw, 



where G(jw) is the closed loop frequency response of the system considered. 
Floyd's procedure for evaluating this integral is to plot the real part of 
the closed loop frequency response, and then approximate the curve by 
a series of straight-line segments. This approximation is then treated as 
a summation of trapezoids. Equation (63) is applied to each trapezoid 
and the resulting time functions are added to obtain c(t). 
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Fig. 32. Geometry of a trapezoid for approximating the real part of response 

function. 

Each particular trapezoid is defined as indicated in Fig. 32. Performing 
the integration indicated in eq. (63) the value for the integral is 



(64) 



2Ai /sin o>it\ /sin A^\ 

IT \ Wit )\ Ait / 



where Ai = Xicoi, the area of the trapezoid, and w\ and Ai are denned by 
the figure. The value of c(t) is then the summation for all the trapezoids. 

* 2A n /sin w n t\ /sin A n £\ 
(65) c(t) = 2, ( — 1 I— — )• 

n=1 T \ w n t / \ A n t J 
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Example. Assume the closed loop frequency response, G(jw), of the 
system to be expressed mathematically as 



(66) 



G(ja) = 



18.72 



[(jo: + l) 2 + l][(j a + 0.6) 2 + 9] 



From this the real part of G(jco) is calculated and plotted as shown in 
Fig. 33. The values used for o>, A, and A of the series in eq. (65) are: 



coi = 



A, = 



1.2 + 0.5 

2 
1.2 - 0.5 



2.0 + 1.2 



3.5 - 2.6 



002 = 



0) 3 = 



A, = 



2.0 - 1.2 



Aa = 



3.5 - 2.6 



4i = lX 0.85 



A 2 = 0.66 X 1.6 



«4 



7.2 + 3.6 



A 3 = 0.66 X 3.05 



3.6 + 3.5 



C0 5 = 



A 4 = 



7.2 - 3.6 



A, = 



3.6 - 3.5 



A 4 = 0.07 X 5.4 



A 5 = 0.07 X 3.55 
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Fig. 33. Real part of response function and approximation. 
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«3. 



1.0 

0.8 

0.6 

0.4 

0.2 



0.6 

0.4 

0.2 



0.2 

-0.4 

-0.6 

-0.8 

0.2 



-0.2 











































® 






















I 




















\ 


















— — 


"V 




















\ 


















© 


\ 




















,\ 






Area between curve 

and u axis is 

© + ©-© + ©-© 


V7Y/< 


\ 




/ 


W 






/ 














i 


H 


/ 
















Wtt 


v//y 


f 




























/ 


4 
































^W 1 - 


s\W 


WW 


s\t 








P 


/ 
















5 J 


[y 

























0.2 0.4 0.6 0.8 

Frequency, radians /second 



1.0 



Fig. 34. Illustrating the trapezoids resulting from the straight-line approximation 

shown in Fig. 33. 

Figure 34 illustrates the trapezoidal approximations used for Fig. 33 
and the foregoing calculations. The evaluation of eq. (65) then becomes 



(67) c(t) = (2/tt) 



/sin 0.855£\ /sin 0.35J' 

0.85 ) 

\ 0.855J A 



0.S5t 



/sin 1.6*\ /sin 0At\ /sin S.05t\ /sin 0.45£\ 

V 1.6* / V 0.4* / ~ ' \ 3.05t J V 0A5t J 

/sin 5At\ /sin 1.8t\ /sin 3.55£\ /sin 0.05* 
+ 0.38 ) ) -0.25 1 

V 5.4* A I. St J \ 3.55* A 0.05i 



The sin x/x tables (see Table 3) may be used to facilitate the evaluation 
of this equation at various values of t. 
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The exact solution, obtained by the inverse Laplace transformation, 
gives this result: 

(G8) 

c(t) = 2.28 exp (-0 sin (t + 5.6°) - 0.761 exp (-0.60 sin (3t + 17°). 

For comparison both eqs. (67) and (68) are plotted in Fig. 35. This is 
the system time response to a unit impulse function. If the response to 

1.0 



0.8 



0.6 



0.4 



0.2 



-0.2 







/'\ 












//~\S 












if \ 










i 




^ 








n 
I 




^ z-— Approximation 




i 




\\ 








l 




\\ 








1 




\\ 








// 




\\ 








i 












il 




\\ 








il 
i 


Exact -""W 








il 




\\ 








il 




\\ 








il 




\ 


\ 




I 








V\ 




il 












// 








\v 




V 






















^^^^r*** 



0.5 1.0 1.5 2.0 2.5 3.0 

Time, seconds 



Fig. 35. Transient response for illustrative problem. 

a step function is desired instead, the graphical integration of the curve 
for the impulse response provides it (Ref. 5). 

Determining Frequency Response from Transient Response 

Quite often the frequency response characteristics of a component or 
system need to be known but it is difficult to introduce a sinusoidal signal 
or to measure magnitude and phase shift of the output. In many cases it 
is much simpler to introduce an impulse or step input; and since time and 
frequency responses are uniquely related, it is possible to obtain the 
frequency response from the transient response. 

There are several approximate methods which have been developed for 
accomplishing this. Floyd's trapezoidal approximation method may be 
used but it yields only the real part of G(joS). To obtain the total vector 
magnitude and phase shift a set of curves such as those presented by 
Bode (Ref. 6) must be used. Other methods have been developed by 
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Bedford and Fredendall, by Teasdale, Brooks and German, and by Samulon 
(Refs. 7, 8, and 9). 

Samulon' s Method. While the approaches vary somewhat the results 
are the same with the exception that Samulon's final equation has a 
"correction" term which makes it more accurate than the others. His 
procedure is presented here. Its basis is: 

Shannon's Sampling Theorem. If a junction c(t) contains no frequen- 
cies higher than f co cycles per second it is completely determined by giving its 
ordinate at a series of points spaced l/(2/ co ) seconds apart. Nearly any 
transient response curve will have some limiting value for its frequency 
spectrum, either due to the properties of the system or the test equipment 
itself. Example. The bandpass of the oscillograph might be the limiting 
item. 

Shannon has also pointed out that such a function, with limited fre- 
quency components, can be exactly synthesized by a sum of sin x/x func- 




t, seconds 
Fig. 36. Use of sin x/x function to approximate transient response. 



tions in a manner indicated in Fig. 36. The equation resulting from this 
approach is 



(69) 



OU») = 



k2/ W, 



W vJ 



exp 



(i W o) 23 B n 



exp (—jam?-) 



sin 



where B n = the increment in the time response curve for a step function 
input, 
to = the frequency of interest, radians/second, 
oi co = the cutoff frequency for the system, radians/second, 
t = the sampling interval is equal to 7r/co co . 
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Equation (69) would be exact if the system response contained no 
frequency components greater than co co . This will never be absolutely 
true in a practical system but good results may be obtained nevertheless. 
In choosing the nominal cutoff frequency, a> co , the attempt should be made 
to estimate the frequency at which the steady-state frequency response is 
attenuated by at least 20 db. A good estimate of u co is ten times co c , the 
crossover frequency, as approximated in Table 1. 

The calculated response will be in error at frequencies lower than the 
w c0 selected if the true response contains higher frequencies. It is there- 
fore desirable to have a frequency characteristic which attenuates rapidly 
above co co selected for calculation. If the system or instrumentation, does 
not provide this attenuation a filter may be added. Samulon states 
"the amount of error, which will be largest near the nominal cutoff frequency, 
co co , will be in general smaller than the amplitude response at co co , provided 
that the response does not rise again above its value at co co for frequencies greater 
than u co ." The calculated frequency response will indicate how valid the 
assumption of the cutoff frequency was. Note that with use of a lower 
co co fewer points must be calculated. 
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Fig. 37. Transient response for illustrative problem. 

Example. Assume a system with a time response to a step input as 
shown in Fig. 37. By assuming a system cutoff frequency of co co of 15.7 
rad per second, 

(70) fco = 15.7/2tt = 2.5 cps. 

By Shannon's theorem the sampling interval should be 

(71) r = l/(2/ co ) = 0.2 sec. 
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By reading the ordinates from the curve at the sampling points Table 2 is 
constructed. For computational convenience the frequency response at 
co = 7r/1.6 will be computed. 

Table 2. Frequency Response Calculated by Samulon's Method for 

CO = 7T/1.6 

.B n exp (— j'conr) 





c(nr) 


B n 


Real 


Imaginary 


TIT 


+ 


+ 


— 


0.2 


0.57 


0.57 


0.527 




0.218 


0.4 


0.93 


0.36 


0.254 




0.254 


0.6 


1.13 


0.20 


0.076 




0.185 


0.8 


1.22 


0.09 






0.090 


1.0 


1.23 


0.01 


0.004 




0.009 


1.2 


1.20 


-0.03 


0.021 


0.021 




1.4 


1.15 


-0.05 


0.046 


0.019 




1.6 


1.09 


-0.06 


0.060 






1.8 


1.04 


-0.05 


0.046 




0.019 


2.0 


1.00 


-0.04 


0.028 




0.028 


2.2 


0.97 


-0.03 


0.011 




0.028 


2.4 


0/95 


-0.02 






0.020 


2.6 


0.94 


-0.01 


0.004 




0.009 


2.8 


0.94 










3.0 


0.95 


0.01 


0.009 


0.004 




3.2 


0.96 


0.01 


0.010 






3.4 


0.97 


0.01 


0.009 




0.004 


3.6 


0.98 


0.01 


0.007 




0.007 


3.8 


0.99 


0.01 


0.004 




0.009 


4.0 


1.00 


0.01 






0.010 


4.2 


1.00 












1.108 - 0.008 0.044 - 0.890 



+ 1.10 -jO.846 

(72) £ Bn exp (- jconr) = 1.10 - ./0.846 = 1.385 /-37.6 . 

The vectors which are shown resolved and added numerically in Table 2 
may be added graphically by simply plotting them end to end. 
The correction terms in eq. (69) will now be computed. 

Magnitude correction: 

(7r/2)(7r/1.6)(l/57r) tt/16 

(73) -±±J± il = = 1.007. 

sin(7r/2)(7r/1.6)(l/57r) sin (tt/16) 

Phase correction: 

(74) exp (icor/2) = exp j(tt/1.6)(0.1) = l/ll.2°. 
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The final result is : 
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<?0«) = 



(tt/2)(o;/co co ) 



exp 0W/2)2j3 n exp (—jamr) 



(75) 



sin (tt/2)(co/co co ) 
= (1.007)(1. 0/11.2 o )(1.38 5/-37.6° ) 
= 1.395 /-26.4 . 

The function chosen as an example is: 

c{t) = 1 — exp ( — 2t) + exp (—t) sin l.5t, 
3.5s 2 + 7s + 6.5 



(76) 
(77) 



(78) 



C(s) = 



(a + 2)[(s + l) 2 + 2.25] 
(1 - 0.538w) 2 + j(1.077w) 



(1 - 0.616w 2 ) +i(1.115w - 0.154<o 3 ) 

The computed point is shown plotted on the exact Cr(jco) curve in Fig. 38. 

i i i i i 1 1 1 
z Computed magnitude 

20 



<u —4 



a -8 



-12 



-16 















1 1 1 1 II 1 1 
, Computed magnitude 
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Fig. 38. Exact response curves and calculated points for example problem. 

The sin x/x values given in Table 3 may be used to aid in computing the 
magnitude correction. Generally, these correction terms will be negligible; 
however, if accuracy is important they should be checked. Samulon 
(Ref. 9) presents a series of tables and nomographs which are useful if 
extensive work of this kind is to be done. 

The previous example illustrates the fact that the number of calculations 
required makes the whole problem rather tedious. If a great amount of 
such work is to be done, a special purpose analog computer may be used 
(Ref. 10). 
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Table 3. A Four-Place Table op sinx/x (Ref. 11) 



X 





1 


2 


3 


4 


5 


6 


7 


8 


9 


0.0 


+ 10000 


10000 


9999 


9999 


9997 


9996 


9994 


9992 


9989 


9987 


0.1 


9983 


9980 


9976 


9972 


9967 


9963 


9957 


9952 


9946 


9940 


0.2 


9933 


9927 


9919 


9912 


9904 


9896 


9889 


9879 


9870 


9860 


0.3 


9851 


9840 


9830 


9820 


9808 


9797 


9785 


9774 


9761 


9748 


0.4 


9735 


9722 


9709 


9695 


9680 


9666 


9651 


9636 


9620 


9605 


0.5 


+9589 


9572 


9555 


9538 


9521 


9503 


9486 


9467 


9449 


9430 


0.6 


9411 


9391 


9372 


9351 


9331 


9311 


9290 


9269 


9247 


9225 


0.7 


9203 


9181 


9158 


9135 


9112 


9089 


9065 


9041 


9016 


8992 


0.8 


8967 


8942 


8916 


8891 


8865 


8839 


8812 


8785 


8758 


8731 


0.9 


8704 


8676 


8648 


8620 


8591 


8562 


8533 


8504 


8474 


8445 


1.0 


+8415 


8384 


8354 


8323 


8292 


8261 


8230 


8198 


8166 


8134 


1.1 


8102 


8069 


8037 


8004 


7970 


7937 


7903 


7870 


7836 


7801 


1.2 


7767 


7732 


7698 


7663 


7627 


7592 


7556 


7520 


7484 


7448 


1.3 


7412 


7375 


7339 


7302 


7265 


7228 


7190 


7153 


7115 


7077 


1.4 


7039 


7001 


6962 


6924 


6885 


6846 


6807 


6768 


6729 


6690 


1.5 


+6650 


6610 


6570 


6530 


6490 


6450 


6410 


6369 


6328 


6288 


1.6 


6247 


6206 


6165 


6124 


6083 


6042 


6000 


5959 


5917 


5875 


1.7 


5833 


5791 


5749 


5707 


5665 


5623 


5580 


5538 


5495 


5453 


1.8 


5410 


5368 


5325 


5282 


5239 


5196 


5153 


5110 


5067 


5024 


1.9 


4981 


4937 


4894 


4851 


4807 


4764 


4720 


4677 


4634 


4590 


2.0 


+4546 


4503 


4459 


4416 


4372 


4329 


4285 


4241 


4198 


4153 


2.1 


4111 


4067 


4023 


3980 


3936 


3893 


3849 


3805 


3762 


3718 


2.2 


3675 


3632 


3588 


3545 


3501 


3458 


3415 


3372 


3328 


3285 


2.3 


3242 


3199 


3156 


3113 


3070 


3028 


2984 


2942 


2899 


2857 


2.4 


2814 


2772 


2730 


2687 


2645 


2603 


2561 


2519 


2477 


2436 


2.5 


+2394 


2352 


2311 


2269 


2228 


2187 


2146 


2105 


2064 


2023 


2.6 


1983 


1942 


1902 


1861 


1821 


1781 


1741 


1702 


1662 


1622 


2.7 


1583 


1544 


1504 


1465 


1427 


1388 


1349 


1311 


1273 


1234 


2.8 


1196 


1159 


1121 


1083 


1046 


1009 


972 


935 


898 


861 


2.9 


825 


789 


753 


717 


681 


646 


610 


575 


540 


505 


3.0 


+470 


436 


402 


368 


334 


300 


266 


233 


200 


167 


3.1 


+134 


+ 102 


+69 


+37 


+5 


-27 


-58 


-90 


-121 


-152 


3.2 


-182 


213 


243 


273 


303 


333 


362 


392 


421 


449 


3.3 


478 


506 


535 


562 


590 


618 


645 


672 


699 


725 


3.4 


752 


778 


804 


829 


855 


880 


905 


930 


954 


978 


X 





1 


2 


3 


4 


5 


6 


7 


8 


9 
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Table 3. A Four-Place Table of sinx/x (Ref. 11) (Continued) 



X 





1 


2 


3 


4 


5 


6 


7 


8 


9 


3.5 


-1002 


1026 


1050 


1073 


1096 


1119 


1141 


1164 


1186 


1208 


3.6 


1229 


1251 


1272 


1293 


1313 


1334 


1354 


1374 


1393 


1413 


3.7 


1432 


1451 


1470 


1488 


1506 


1524 


1542 


1559 


1576 


1593 


3.8 


1610 


1627 


1643 


1659 


1675 


1690 


1705 


1720 


1735 


1749 


3.9 


1764 


1777 


1791 


1805 


1818 


1831 


1844 


1856 


1868 


1880 


4.0 


-1892 


1903 


1915 


1926 


1936 


1947 


1957 


1967 


1977 


1987 


4.1 


1996 


2005 


2014 


2022 


2030 


2039 


2046 


2054 


2061 


2068 


4.2 


2075 


2082 


2088 


2094 


2100 


2106 


2111 


2116 


2121 


2126 


4.3 


2131 


2135 


2139 


2143 


2146 


2150 


2153 


2156 


2158 


2161 


4.4 


2163 


2165 


2166 


2168 


2169 


2170 


2171 


2172 


2172 


2172 


4.5 


-2172 


2172 


2172 


2171 


2170 


2169 


2168 


2166 


2164 


2162 


4.6 


2160 


2158 


2155 


2152 


2150 


2146 


2143 


2139 


2136 


2132 


4.7 


2127 


2123 


2119 


2114 


2109 


2104 


2098 


2093 


2087 


2081 


4.8 


2075 


2069 


2063 


2056 


2049 


2042 


2035 


2028 


2020 


2013 


4.9 


2005 


1997 


1989 


1981 


1972 


1963 


1955 


1946 


1937 


1927 


5.0 


-1918 


1908 


1899 


1889 


1879 


1868 


1858 


1848 


1837 


1826 


5.1 


1815 


1804 


1793 


1782 


1770 


1759 


1747 


1735 


1723 


1711 


5.2 


1699 


1687 


1674 


1662 


1649 


1636 


1623 


1610 


1597 


1584 


5.3 


1570 


1557 


1543 


1530 


1516 


1502 


1488 


1474 


1460 


1445 


5.4 


1431 


1417 


1402 


1387 


1373 


1358 


1343 


1328 


1313 


1298 


5.5 


-1283 


1268 


1252 


1237 


1221 


1206 


1190 


1175 


1159 


1143 


5.6 


1127 


1111 


1095 


1079 


1063 


1047 


1031 


1015 


999 


982 


5.7 


966 


950 


933 


917 


900 


884 


867 


851 


834 


818 


5.8 


800 


784 


768 


751 


734 


718 


701 


684 


667 


650 


5.9 


634 


617 


600 


583 


567 


550 


533 


516 


499 


482 


6.0 


-466 


449 


432 


416 


399 


382 


365 


348 


332 


315 


6.1 


299 


282 


265 


249 


232 


216 


200 


183 


167 


150 


6.2 


-134 


-118 


-102 


-85 


-69 


-53 


-37 


-21 


-5 


+ 11 


6.3 


+27 


43 


58 


74 


90 


105 


121 


136 


152 


167 


6.4 


182 


197 


212 


227 


242 


257 


272 


287 


302 


316 


6.5 


+331 


346 


360 


374 


388 


403 


417 


431 


445 


458 


6.6 


472 


486 


499 


513 


526 


539 


552 


566 


579 


591 


6.7 


604 


617 


630 


642 


654 


667 


679 


691 


703 


715 


6.8 


727 


738 


750 


761 


773 


784 


795 


806 


817 


828 


6.9 


838 


849 


859 


870 


880 


890 


900 


910 


919 


929 


X 





1 


2 


3 


4 


5 


6 


7 


8 


9 
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Table 3. A Four-Place Table of sinx/x (Ref. 11) {Continued) 



X 





1 


2 


3 


4 


5 


6 


7 


8 


9 


7.0 


+939 


948 


957 


966 


975 


984 


993 


1002 


1010 


1019 


7.1 


1027 


1035 


1043 


1051 


1058 


1066 


1074 


1081 


1088 


1095 


7.2 


1102 


1109 


1116 


1123 


1129 


1135 


1142 


1148 


1153 


1159 


7.3 


1165 


1171 


1176 


1181 


1186 


1191 


1196 


1201 


1206 


1210 


7.4 


1214 


1219 


1223 


1227 


1231 


1234 


123S 


1241 


1244 


1248 


7.5 


+ 1251 


1254 


1256 


1259 


1261 


1264 


1266 


1268 


1270 


1272 


7.6 


1274 


1275 


1277 


1278 


1279 


1280 


1281 


1282 


1282 


1283 


7.7 


1283 


1284 


1284 


1284 


1284 


1283 


1283 


1282 


1282 


12S1 


7.8 


1280 


1279 


1278 


1277 


1275 


1274 


1272 


1270 


1269 


1267 


7.9 


1264 


1262 


1259 


1257 


1255 


1252 


1249 


1246 


1243 


1240 


8.0 


+ 1237 


1233 


1230 


1226 


1222 


1218 


1214 


1210 


1206 


1202 


8.1 


1197 


1193 


1188 


1183 


1179 


1174 


1169 


1163 


1158 


1153 


8.2 


1147 


1142 


1136 


1130 


1124 


1118 


1112 


1106 


1100 


1093 


8.3 


1087 


1080 


1074 


1067 


1060 


1053 


1046 


1039 


1032 


1025 


8.4 


1017 


1010 


1002 


995 


987 


979 


972 


964 


956 


948 


8.5 


+939 


931 


923 


915 


906 


898 


889 


880 


872 


863 


8.6 


854 


845 


836 


827 


818 


809 


800 


790 


781 


771 


8.7 


762 


752 


743 


733 


724 


714 


704 


694 


684 


675 


8.8 


665 


655 


645 


635 


625 


614 


604 


594 


584 


573 


8.9 


563 


552 


542 


532 


521 


511 


500 


490 


479 


469 


9.0 


+458 


447 


437 


426 


415 


404 


394 


383 


372 


361 


9.1 


351 


340 


329 


318 


307 


296 


286 


275 


264 


253 


9.2 


242 


231 


220 


210 


199 


188 


177 


166 


156 


145 


9.3 


134 


123 


112 


101 


91 


80 


69 


58 


48 


37 


9.4 


+26 


+ 16 


+5 


-6 


-16 


-27 


-37 


-48 


-58 


-69 


9.5 


-79 


89 


100 


110 


120 


131 


141 


151 


161 


172 


9.6 


182 


192 


202 


212 


222 


231 


241 


251 


261 


271 


9.7 


280 


290 


299 


309 


318 


328 


337 


346 


356 


365 


9.8 


374 


383 


392 


401 


410 


419 


428 


436 


445 


454 


9.9 


462 


471 


479 


487 


496 


504 


512 


520 


528 


536 


10.0 


-544 


552 


560 


567 


575 


582 


590 


597 


604 


612 


10.1 


619 


626 


633 


640 


647 


653 


660 


667 


673 


680 


10.2 


686 


692 


699 


705 


711 


717 


723 


728 


734 


740 


10.3 


745 


751 


756 


761 


767 


772 


777 


782 


787 


791 


10.4 


796 


801 


805 


809 


814 


818 


822 


826 


830 


834 


X 





1 


2 


3 


4 


5 


6 


7 


8 


9 
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Table 3. A Four-Place Table of sinx/x (Ref. 11) {Continued) 



X 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10.5 


-838 


842 


845 


849 


852 


855 


859 


862 


865 


868 


10. G 


871 


873 


876 


879 


881 


883 


886 


888 


890 


892 


10.7 


894 


896 


898 


899 


901 


902 


904 


905 


906 


907 


10.8 


908 


909 


910 


911 


911 


912 


912 


913 


913 


913 


10.9 


913 


913 


913 


913 


913 


912 


912 


911 


911 


910 


11.0 


-909 


908 


907 


906 


905 


904 


902 


901 


899 


898 


11.1 


896 


894 


892 


890 


888 


886 


884 


882 


879 


877 


11.2 


874 


872 


869 


866 


863 


860 


857 


854 


851 


848 


11.3 


844 


841 


837 


834 


830 


826 


822 


819 


815 


811 


11.4 


806 


802 


798 


794 


789 


785 


780 


776 


771 


766 


11.5 


-761 


756 


751 


746 


741 


736 


731 


726 


720 


715 


11.6 


709 


704 


698 


693 


687 


681 


675 


669 


663 


657 


11.7 


651 


645 


639 


633 


626 


620 


614 


607 


601 


594 


11.8 


588 


581 


574 


568 


561 


554 


547 


540 


533 


526 


11.9 


519 


512 


505 


498 


491 


484 


476 


469 


462 


454 


12.0 


-447 


440 


432 


425 


417 


410 


402 


395 


387 


379 


12.1 


372 


364 


356 


348 


341 


333 


325 


317 


309 


301 


12.2 


294 


286 


278 


270 


262 


254 


246 


238 


230 


222 


12.3 


214 


206 


198 


190 


182 


174 


166 


158 


150 


142 


12.4 


134 


125 


117 


109 


101 


93 


85 


77 


69 


61 


12.5 


-53 


-45 


-37 


-29 


-21 


-13 


-5 


+3 


+ 11 


+ 19 


12.6 


+27 


35 


42 


50 


58 


66 


74 


82 


89 


97 


12.7 


105 


113 


120 


128 


136 


143 


151 


158 


166 


173 


12.8 


181 


188 


196 


203 


210 


218 


225 


232 


240 


247 


12.9 


254 


261 


268 


275 


282 


289 


296 


303 


310 


316 


13.0 


+323 


330 


337 


343 


350 


356 


363 


369 


376 


382 


13.1 


388 


395 


401 


407 


413 


419 


425 


431 


437 


443 


13.2 


448 


454 


460 


466 


471 


477 


482 


488 


493 


498 


13.3 


503 


509 


514 


519 


524 


529 


534 


538 


543 


548 


13.4 


552 


557 


562 


566 


570 


575 


579 


583 


587 


591 


13.5 


+595 


599 


603 


607 


611 


614 


618 


622 


625 


628 


13.6 


632 


635 


638 


641 


644 


647 


650 


653 


656 


659 


13.7 


661 


664 


666 


669 


671 


673 


676 


678 


680 


682 


13.8 


684 


686 


688 


689 


691 


692 


694 


695 


697 


698 


13.9 


699 


700 


702 


703 


703 


704 


705 


706 


706 


707 


X 





1 


2 


3 


4 


5 


6 


7 


8 


9 
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Table 3. A Four-Place Table of sinx/x (Ref. 11) (Continued) 



X 





1 


2 


3 


4 


5 


6 


7 


8 


9 


14.0 


+708 


708 


708 


709 


709 


709 


709 


709 


709 


709 


14.1 


709 


708 


708 


708 


707 


707 


706 


705 


705 


704 


14.2 


703 


702 


701 


700 


699 


697 


696 


695 


693 


692 


14.3 


690 


688 


687 


685 


683 


681 


679 


677 


675 


673 


14.4 


671 


668 


666 


663 


661 


658 


656 


653 


650 


648 


14.5 


+645 


642 


639 


636 


633 


630 


626 


623 


620 


616 


14.6 


613 


609 


606 


602 


599 


595 


591 


587 


583 


579 


14.7 


575 


571 


567 


563 


559 


555 


550 


546 


542 


537 


14.8 


533 


528 


524 


519 


514 


509 


505 


500 


495 


490 


14.9 


485 


480 


475 


470 


465 


460 


455 


449 


444 


439 


15.0 


+434 


428 


423 


417 


412 


406 


401 


395 


390 


384 


15.1 


378 


373 


367 


361 


355 


349 


344 


338 


332 


326 


15.2 


320 


314 


308 


302 


296 


290 


284 


278 


272 


265 


15.3 


259 


253 


247 


241 


234 


228 


222 


216 


209 


203 


15.4 


197 


190 


184 


178 


171 


165 


159 


152 


146 


140 


15.5 


+ 133 


127 


120 


114 


108 


101 


95 


88 


82 


76 


15.6 


69 


63 


56 


50 


43 


37 


31 


24 


18 


11 


15.7 


+5 


-1 


-8 


-14 


-20 


-27 


-33 


-39 


-46 


-52 


15.8 


58 


64 


71 


77 


83 


89 


95 


102 


108 


114 


15.9 


120 


126 


132 


138 


144 


150 


156 


162 


168 


174 


16.0 


-180 


186 


192 


197 


203 


209 


215 


220 


226 


232 


16.1 


237 


243 


248 


254 


259 


265 


270 


276 


281 


286 


16.2 


292 


297 


302 


307 


312 


318 


323 


328 


333 


337 


16.3 


342 


347 


352 


357 


362 


366 


371 


376 


380 


385 


16.4 


389 


393 


398 


402 


407 


411 


415 


419 


423 


427 


16.5 


-431 


435 


439 


443 


447 


451 


454 


458 


462 


465 


16.6 


469 


472 


476 


479 


482 


486 


489 


492 


495 


498 


16.7 


501 


504 


507 


510 


513 


515 


518 


521 


523 


526 


16.8 


528 


531 


533 


535 


538 


540 


542 


544 


546 


548 


16.9 


550 


552 


553 


555 


557 


558 


560 


561 


563 


564 


17.0 


-566 


567 


568 


569 


570 


571 


572 


573 


574 


575 


17.1 


575 


576 


577 


577 


578 


578 


579 


579 


579 


579 


17.2 


580 


580 


580 


580 


580 


579 


579 


579 


579 


578 


17.3 


578 


577 


577 


576 


576 


575 


574 


573 


572 


571 


17.4 


570 


569 


568 


567 


566 


565 


563 


562 


561 


559 


X 





1 


2 


3 


4 


5 


6 


7 


8 


9 
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Table 3. A Four-Place Table of sinx/x (Ref. 11) (Continued) 



X 





1 


2 


3 


4 


5 


6 


7 


8 


9 


17.5 


-557 


556 


554 


553 


551 


549 


547 


545 


543 


541 


17.6 


539 


537 


535 


533 


530 


528 


526 


523 


521 


518 


17.7 


516 


513 


510 


508 


505 


502 


499 


496 


493 


490 


17.8 


487 


484 


481 


478 


475 


471 


468 


465 


461 


458 


17.9 


454 


451 


447 


444 


440 


436 


433 


429 


425 


421 


18.0 


-417 


413 


409 


405 


401 


397 


393 


389 


385 


381 


18.1 


376 


372 


368 


364 


359 


355 


350 


346 


341 


337 


18.2 


332 


328 


323 


319 


314 


309 


304 


300 


295 


290 


18.3 


285 


281 


276 


271 


266 


261 


256 


251 


246 


241 


18.4 


236 


231 


226 


221 


216 


211 


206 


201 


195 


190 


18.5 


-185 


180 


175 


170 


164 


159 


154 


149 


143 


138 


18.6 


133 


128 


122 


117 


112 


106 


101 


96 


90 


85 


18.7 


80 


74 


69 


64 


58 


53 


48 


42 


37 


32 


18.8 


-26 


-21 


-16 


-10 


-5 


+0 


+6 


+ 11 


+ 16 


+21 


18.9 


+27 


32 


37 


42 


48 


53 


58 


63 


68 


74 


19.0 


+79 


84 


89 


94 


99 


104 


110 


115 


120 


125 


19.1 


130 


135 


140 


145 


150 


155 


159 


164 


169 


174 


19.2 


179 


184 


188 


193 


198 


202 


207 


212 


216 


221 


19.3 


226 


230 


235 


239 


244 


248 


252 


257 


261 


265 


19.4 


270 


274 


278 


282 


286 


290 


295 


299 


303 


307 


19.5 


+311 


314 


318 


322 


326 


330 


333 


337 


341 


344 


19.6 


348 


351 


355 


358 


362 


365 


369 


372 


375 


378 


19.7 


382 


385 


388 


391 


394 


397 


400 


403 


405 


408 


19.8 


411 


414 


416 


419 


422 


424 


427 


429 


431 


434 


19.9 


436 


438 


440 


443 


445 


447 


449 


451 


453 


455 


X 





1 


2 


3 


4 


5 


6 


7 


8 


9 
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Table 3. A Four-Place Table op sinx/x (Ref. 11) (Continued) 



X 





2 


4 


6 


8 


X 





2 


4 


6 


8 


20.0 


+456 


460 


463 


466 


469 


23.5 


-425 


425 


425 


424 


424 


20.1 


472 


475 


477 


479 


481 


23.6 


423 


423 


422 


421 


419 


20.2 


483 


485 


486 


487 


488 


23.7 


418 


416 


415 


413 


411 


20.6 


489 


490 


490 


490 


490 


23.8 


408 


406 


403 


401 


398 


20.4 


490 


490 


489 


488 


487 


23.9 


395 


392 


388 


385 


381 


20.5 


+486 


485 


483 


482 


480 


24.0 


-377 


373 


369 


365 


361 


20.6 


478 


475 


473 


470 


467 


24.1 


356 


352 


347 


342 


337 


20.7 


464 


461 


458 


454 


450 


24.2 


332 


327 


321 


316 


310 


20.8 


447 


442 


438 


434 


429 


24.3 


304 


299 


293 


287 


280 


20.9 


424 


420 


415 


409 


404 


24.4 


274 


268 


261 


255 


248 


21.0 


+398 


393 


387 


381 


375 


24.5 


-241 


235 


228 


221 


214 


21.1 


369 


362 


356 


349 


342 


24.6 


206 


199 


192 


185 


177 


21.2 


335 


328 


321 


314 


307 


24.7 


170 


162 


155 


147 


139 


21.3 


299 


292 


284 


276 


268 


24.8 


132 


124 


116 


108 


100 


21.4 


260 


252 


244 


236 


228 


24.9 


93 


85 


77 


69 


61 


21.5 


+219 


211 


202 


194 


185 


25.0 


-53 


45 


37 


29 


21 


21.6 


176 


168 


159 


150 


141 


25.1 


-13 


-5 


+3 


+ 11 


+ 19 


21.7 


132 


123 


114 


105 


96 


25.2 


+27 


35 


42 


50 


58 


21.8 


87 


78 


69 


60 


51 


25.3 


66 


74 


81 


89 


96 


21.9 


42 


32 


23 


14 


5 


25.4 


104 


111 


119 


126 


134 


22.0 


-4 


13 


22 


31 


40 


25.5 


+ 141 


148 


155 


162 


169 


22.1 


49 


58 


67 


76 


85 


25.6 


176 


183 


189 


196 


203 


22.2 


93 


102 


111 


119 


128 


25.7 


209 


215 


222 


228 


234 


22.3 


136 


145 


153 


161 


169 


25.8 


240 


246 


251 


257 


263 


22.4 


178 


185 


193 


201 


209 


25.9 


268 


273 


278 


284 


288 


22.5 


-217 


224 


231 


239 


246 


26.0 


+293 


298 


303 


307 


311 


22.6 


253 


260 


267 


274 


280 


26.1 


315 


320 


323 


327 


331 


22.7 


287 


293 


299 


305 


311 


26.2 


334 


338 


341 


344 


347 


22.8 


317 


323 


329 


334 


339 


26.3 


350 


352 


355 


357 


359 


22.9 


344 


349 


354 


359 


364 


26.4 


361 


363 


365 


367 


368 


23.0 


-368 


372 


376 


380 


384 


26.5 


+370 


371 


372 


373 


373 


23.1 


388 


391 


394 


397 


400 


26.6 


374 


374 


375 


375 


375 


23.2 


403 


406 


408 


410 


413 


26.7 


375 


374 


374 


373 


372 


23.3 


415 


416 


418 


419 


421 


26.8 


371 


370 


369 


368 


366 


23.4 


422 


423 


423 


424 


424 


26.9 


365 


363 


361 


359 


357 


X 





2 


4 


6 


8 


X 





2 


4 


6 


8 
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Table 3. A Four-Place Table of sinx/x (Ref. 11) (Continued) 



X 





2 


4 


6 


8 


X 





2 


4 


6 


8 


27.0 


+354 


352 


349 


346 


343 


30.5 


-260 


256 


252 


247 


243 


27.1 


340 


337 


334 


331 


327 


30.6 


238 


233 


229 


224 


219 


27.2 


323 


319 


316 


312 


307 


30.7 


214 


209 


204 


198 


193 


27.3 


303 


299 


294 


290 


285 


30.8 


188 


182 


177 


171 


165 


27.4 


280 


275 


270 


265 


260 


30.9 


160 


154 


148 


142 


136 


27.5 


4-254 


249 


243 


238 


232 


31.0 


-130 


124 


118 


112 


106 


27.6 


226 


220 


214 


208 


202 


31.1 


100 


94 


87 


81 


75 


27.7 


196 


190 


184 


177 


171 


31.2 


69 


62 


56 


50 


43 


27.8 


164 


158 


151 


145 


138 


31.3 


37 


31 


24 


18 


11 


27.9 


131 


124 


117 


111 


104 


31.4 


-5 


+ 1 


+8 


+ 14 


+20 


28.0 


4-97 


90 


83 


76 


69 


31.5 


+27 


33 


39 


45 


52 


28.1 


62 


55 


48 


41 


34 


31.6 


58 


64 


70 


76 


82 


28.2 


4-26 


+ 19 


+ 12 


+5 


-2 


31.7 


88 


94 


100 


106 


112 


28.3 


-9 


16 


23 


30 


37 


31.8 


118 


124 


129 


135 


140 


28.4 


44 


51 


58 


65 


72 


31.9 


146 


151 


157 


162 


167 


28.5 


-79 


85 


92 


99 


105 


32.0 


+ 172 


177 


182 


187 


192 


28.6 


112 


118 


125 


131 


138 


32.1 


197 


202 


206 


211 


215 


28.7 


144 


150 


156 


162 


168 


32.2 


219 


224 


228 


232 


236 


28.8 


174 


180 


186 


192 


197 


32.3 


239 


243 


247 


250 


254 


28.9 


203 


208 


213 


219 


224 


32.4 


257 


260 


263 


266 


269 


29.0 


-229 


234 


239 


243 


248 


32.5 


+272 


275 


277 


280 


282 


29.1 


253 


257 


261 


266 


270 


32.6 


284 


286 


288 


290 


292 


29.2 


274 


278 


281 


285 


288 


32.7 


293 


295 


296 


297 


299 


29.3 


292 


295 


298 


301 


304 


32.8 


300 


300 


301 


302 


302 


29.4 


307 


310 


312 


315 


317 


32.9 


303 


303 


303 


303 


303 


29.5 


-319 


321 


323 


325 


326 


33.0 


+303 


303 


302 


302 


301 


29.6 


328 


329 


330 


331 


332 


33.1 


300 


299 


298 


297 


296 


29.7 


333 


334 


334 


335 


335 


33.2 


294 


293 


291 


290 


288 


29.8 


335 


335 


335 


335 


334 


33.3 


286 


284 


281 


279 


277 


29.9 


334 


333 


332 


332 


331 


33.4 


274 


272 


269 


266 


263 


30.0 


-329 


328 


327 


325 


323 


33.5 


+260 


257 


254 


250 


247 


30.1 


321 


320 


317 


315 


313 


33.6 


243 


240 


236 


232 


228 


30.2 


311 


308 


305 


302 


300 


33.7 


224 


220 


216 


212 


208 


30.3 


296 


293 


290 


287 


283 


33.8 


203 


199 


194 


190 


185 


30.4 


280 


276 


272 


268 


264 


33.9 


180 


175 


171 


166 


161 


X 





2 


4 


6 


8 


X 





2 


4 


6 


8 
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Table 3. A Four-Place Table op sinx/x (Ref. 11) (Continued) 



X 





2 


4 


6 


8 


X 





2 


4 


6 


8 


34.0 


+156 


151 


145 


140 


135 


37.0 


-174 


170 


165 


161 


156 


34.1 


130 


124 


119 


113 


108 


37.1 


152 


147 


143 


138 


133 


34.2 


102 


97 


91 


86 


80 


37.2 


129 


124 


119 


114 


109 


34.3 


74 


69 


63 


57 


51 


37.3 


104 


99 


94 


89 


84 


34.4 


46 


40 


34 


28 


22 


37.4 


79 


74 


68 


63 


58 


34.5 


+ 17 


+ 11 


+5 


-1 


-7 


37.5 


-53 


47 


42 


37 


32 


34.6 


-12 


18 


24 


30 


35 


37.6 


26 


21 


16 


10 


5 


34.7 


41 


47 


52 


58 


63 


37.7 


+0 


6 


11 


16 


21 


34.8 


69 


75 


80 


85 


91 


37.8 


27 


32 


37 


42 


47 


34.9 


96 


102 


107 


112 


117 


37.9 


53 


58 


63 


68 


73 


35.0 


-122 


127 


132 


137 


142 


38.0 


+78 


83 


88 


93 


98 


35.1 


147 


152 


157 


161 


166 


38.1 


102 


107 


112 


117 


121 


35.2 


170 


175 


179 


183 


187 


38.2 


126 


130 


135 


139 


143 


35.3 


192 


196 


199 


203 


207 


38.3 


148 


152 


156 


160 


164 


35.4 


211 


214 


218 


221 


225 


38.4 


168 


172 


176 


179 


183 


35.5 


-228 


231 


234 


237 


240 


38.5 


+ 186 


190 


193 


197 


200 


35.6 


243 


245 


248 


250 


253 


38.6 


203 


206 


209 


212 


215 


35.7 


255 


257 


259 


261 


263 


38.7 


218 


220 


223 


225 


228 


35.8 


264 


266 


268 


269 


270 


38.8 


230 


232 


234 


236 


238 


35.9 


271 


272 


273 


274 


275 


38.9 


240 


241 


243 


244 


246 


36.0 


-275 


276 


276 


277 


277 


39.0 


+247 


248 


249 


250 


251 


36.1 


277 


277 


277 


276 


276 


39.1 


252 


253 


253 


254 


254 


36.2 


276 


275 


274 


273 


272 


39.2 


254 


255 


255 


255 


255 


36.3 


271 


270 


269 


268 


266 


39.3 


254 


254 


254 


253 


252 


36.4 


265 


263 


261 


259 


257 


39.4 


252 


251 


250 


249 


248 


36.5 


-255 


253 


251 


248 


246 


39.5 


+246 


245 


244 


242 


241 


36.6 


243 


241 


238 


235 


232 


39.6 


239 


237 


235 


233 


231 


36.7 


229 


226 


223 


220 


216 


39.7 


229 


227 


224 


222 


219 


36.8 


213 


209 


206 


202 


198 


39.8 


217 


214 


211 


208 


206 


36.9 


194 


190 


186 


182 


178 


39.9 


203 


199 


196 


193 


190 


X 





2 


4 


6 


8 


X 





2 


4 


6 


8 
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1. DESIGN CRITERIA AND TECHNIQUES 

The first step in the design of a feedback control system is the selection 
of a suitable power element with sufficient torque, or force, speed, and 
power rating to drive the load. Once the selection of a power element 
with known characteristics has been made, the signal devices, amplifiers, 
and stabilizing components have to be chosen with such characteristics 
that make the entire feedback control system meet system requirements of 
accuracy, speed of response, and stability. This chapter is devoted to the 
synthesis of required characteristics of these compensating components 
and the presentation of characteristics of practical control system com- 
ponents. Section 1 derives feedback control system characteristics from 
system specifications. 

Synthesis of Log Magnitude Diagram from System Requirements 

Low- Frequency Portion: Static Error Coefficients. Error coef- 
ficients are one of the most common means of specifying control system 

23-01 
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performance. These coefficients are figures of merit, the higher the coef- 
ficient, the smaller the control system error in achieving a required 
output. 

The static error coefficients are defined as the ratio of the constant out- 
put required (position, velocity, or acceleration) to the control system 
error required to achieve that output. The types of control system and 
the static error coefficient associated with each type are summarized in 
Chap. 20. (See also Ref. 1, Chap. 8.) 

The static error coefficients influence the log magnitude diagram in an 
easily visualized way and lead to a method of control system classifica- 



db/decade 



- 20 log K p 




co = K, l 



-40 db/decade 
Type 2 system 



20 log K a 




co=l 



Fig. 1. Sample log magnitude diagrams showing influence of static error coefficients. 



tion. For example, a control system with a transfer function that ap- 
proaches K p , a constant, at low frequencies (open loop transfer function, 
G(s) approaches K p as s approaches 0) will have a log magnitude diagram 
which has zero slope at low frequencies. Such a system is called a type 
system (0 slope at low frequencies) and can follow a steady input, r , 
with an error of r /(l + K p ). If K p is large, the error will be small. How- 
ever, if a velocity signal, r = v t, is applied, the error will continue to in- 
crease with time. For a system to follow such a signal with small error, 
a type 1 system is required which has a transfer function at low frequency of 
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K v /jo) (limit sG(s) s=0 = K v ) and an initial slope on the log magnitude 
diagram of — 20 db per decade. A type 1 system would follow the con- 
stant velocity input with an error of only Vq/K v . Similarly, a type 2 system 
(K a /(ju>) 2 transfer function giving a slope of 2 ( — 20) = —40 db per decade 
at low frequencies) is required to follow a constant acceleration input with 
moderate error. 

The type of system determines the shape of the log magnitude diagram 
at low frequencies and the gain magnitude of this portion of the diagram 
is determined by the static error coefficients. The intersection of the 
extensions of the initial log magnitude diagram slope with the co = 1 line 
is at the value 20 log K p , 20 log K v , or 20 log K a , as the case may be. The 
intersection of the extensions of the initial slope with the 0-db axis also 
has significance as shown in Fig. 1. 

Low-Frequency Portion : Dynamic Error Coefficients. In addition 
to the steady-state characteristics, expressible in terms of the static error 
coefficients, it is often desirable to specify control system errors during a 
transient by means of dynamic error coefficients, defined in Chap. 20. 
That is, 

1 1 1 1 

e = — r H r ^ f -\ "r + • • • » 

K Ki K 2 K 3 

where r, f, f are successive derivatives of the input time function and 
K , K\, K 2 , etc., are the dynamic error coefficients. The above relation 
is valid during time intervals in a transient which are far displaced in 
time from a discontinuity in the input function, r, and its derivatives. 
The above equation converges quickly to useful values for slowly changing 
input functions for which the higher order derivatives are small relative to 
the lower order terms. The coefficients can be evaluated by straightforward 
Laplace transform techniques, as given in Chap. 20. That is, 



1 1 d n 
= — lim 



E 

(s) 



K n n\ s ^ods n IR 

Some of the error coefficients, evaluated in this way, will be found 
identical to the static coefficients of the previous paragraph. However, 
additional coefficients will also be determined. The composition of these 
generalized error coefficients can be seen from a general control system 
transfer function (see Ref. 4). 

E(s) n + nis + n 2 s 2 + n 3 s 3 + • • • 



R(s) 1 + dts + d 2 s 2 + d 3 s 3 + 
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The dynamic error coefficients for this system are: 
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— = n » 
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j j=k— 1 -^ 

— = n k — J^ — d(k-j) . 
-ft-fc i=o ^i 

The dynamic error coefficients in general are composed of the gain term 
in combination with various sums and products of the system time con- 
stants. These coefficients are readily calculable and are valuable for 
analysis purposes for a system of known transfer function. However, 
they are not very useful in synthesizing the log magnitude diagram from 
system requirements, because each of the coefficients is composed of a 
number of parameters of the system characteristics. For this synthesis 
work, a more direct procedure is outlined in the next paragraph. 

Low-Frequency Portion: Transient Curve Fitting Procedure. A 
curve fitting procedure (Ref. 2) by which certain system error require- 
ments are transformed directly to log magnitude values which the log 
magnitude diagram must exceed is useful. In this method, the expected 
transient input signals are matched by sinusoids. The principle is that 
if a control system can follow with small error sine wave inputs with 
amplitude, velocity, and acceleration components as great as those of 
the transient input, then it can follow the transient with small error. The 
worst transients that the control system will be expected to follow are 
presumably known, either in graphical or analytical form, together with 
the allowable errors during these transients. These transient time functions 
are plotted and fitted as closely as possible in various places with sine 
waves as indicated in Fig. 2. The amplitudes of these sine waves are Ai, 
A 2 , A 3 , etc., with frequencies coi, a) 2 , co 3 . A x /E, A 2 /E, A 3 /E, etc., are the 
required gain magnitudes of the log magnitude diagram at a>i, w 2 , o) 3 if 
E is the allowable control system error. These points are shown in Fig. 3. 
The required log magnitude diagram must be above these points. 

It is important to fit the input transient at several places, such as peaks 
and maximum slope points, so that broad coverage of requirements is 
established by several points on the log magnitude diagram. Sometimes 
it is advantageous to take the derivative of the input transient and fit it 
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with sine waves of amplitude V\, V 2 , etc., at frequencies a>i, co 2 . These 
fits will establish points on the log magnitude diagram of magnitude 
Vi/uiE, V 2 /c0 2 E, etc. The procedure can be extended to higher deriva- 
tives also. 



fit) 



te 



Input time 
function 




r 2 — ^-H H r 3-H 



(«l = f?) 



Fig. 2. Construction illustrating curve fitting procedure. 



Diagram must be above 

points obtained by curve 

fitting 




Fig. 3. Log magnitude points obtained from Fig. 2. 

Mid-Frequency Portion of the Log Magnitude Diagram. Once 
the gain of the entire control system has been set so as to meet the require- 
ments as outlined in the preceding paragraphs, it is usually desirable to 
reduce the system gain effective at the higher frequencies in order to reduce 
the susceptibility of the system to noise and other extraneous signals. 
However, this reduction in gain has to be achieved in such a manner that 
the system has the required stability. As explained in Chap. 21, stability 
may be assured by requiring the log magnitude diagram to have a slope 
of —20 db per decade in the vicinity of the crossover frequency. To 
obtain adequate stability, this —20 db per decade slope should extend for 
a frequency range of a decade or more. The use of the log magnitude- 
angle chart (Nichols chart) provides a measure of stability in terms of the 
maximum M of the closed loop frequency response. Such charts are given 
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in Chap. 21. To indicate approximate magnitudes, Fig. 4 shows the 
maximum M that could possibly be obtained for a particular minimum 
value of phase margin. 

Often it is convenient to express the degree of stability, or damping, of 
a system by means of a damping factor. Strictly, a damping factor can 
be applied only to a system that can be described by a second order linear 
differential equation with constant coefficients, but it is frequently applied 
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Fig. 4. Maximum peak of frequency response versus minimum phase margin. 

to higher order systems. When the response is determined largely by 
two complex roots, which is fairly common, the closed loop response is 
characterized by a zero slope region of approximately unity gain at low 
frequencies followed by a resonant peak in the vicinity of crossover of the 
open loop. At frequencies above the resonant peak, the slope changes to 
—40 db per decade and then usually to even greater negative slopes. 
Thus for the frequency region from zero to somewhat above the resonant 
peak, many systems have much the same frequency characteristic as a 
second order system. For such a system the height of the resonant peak, 
when expressed as a numeric ratio, M m , determines the damping factor, f , 
by the equation: 

1 
M m = valid: < f < 0.707. 

2rVi - f 
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Frequently it is convenient to measure the magnitude of the frequency 
response, M c , at the corner frequency. For such a measurement, the 
damping factor is 

1 



The damping of oscillations in a physical system is a function of the 
damping factor. When a system is excited by a unit step function, the 
magnitude of the first and successive overshoots is determined by the 
damping factor as shown in Fig. 5. See also Chap. 20. 
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Fig. 5. Overshoot magnitude versus damping factor for unit step function applied to 

second order system. 
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High-Frequency Portion. At frequencies higher than the crossover 
frequency, the log magnitude diagram should be attenuated as rapidly 
as possible, consistent with stability and damping requirements, in order 
to reduce noise. In systems in which there is a relatively large amount 
of noise signal present, the noise error, E n , in the control system output 
may be larger than the system errors, E s , that were considered in the pre- 
ceding paragraphs, even when the high-frequency portion of the log 
magnitude diagram is attenuated. If the noise has significant high- 
frequency components, it will be profitable to lower co c , the crossover 
frequency, and the gain at frequencies below u c in order to reduce E n . 
This procedure increases E s , but the overall root mean square error may 
be reduced. The usual criterion in such cases is to minimize 

E ima = (E n 2 + E S 2 ) V2 - 

See Chap. 24 for a complete treatment of noise evaluation and system 
optimization. 

Figure 6 shows a sample log magnitude diagram synthesized by the 
methods described above. 
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Fig. 6. Sample log magnitude diagram summarizing synthesis procedure. 



Performance Charts Relating Transient and Frequency Response. 

The methods of the previous paragraphs will be found straightforward in 
synthesizing the low- and medium-frequency portions of the log magni- 
tude diagram from system requirements. The degree of stability and the 
nature of the transient response often can be more easily obtained by means 
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of the design charts relating frequency response and transient response as 
given in Chap. 22. 

Load Disturbances. The previous discussion has been concerned with 
obtaining the proper overall open loop gain and frequency characteristics. 
In many practical systems, not only the overall characteristics but also 
the characteristics and order of the individual elements of the system 
become important. The latter situation occurs whenever there are ad- 
ditional or extraneous inputs acting on the system in addition to the 
main controlling signal input. Examples of the extraneous inputs are (a) 
torque changes in the load on the power element of a speed control system 
in which a voltage, proportional to derived speed, is the main input signal 
and (6) electrical pickup of alternating voltage signals from power supplies 
in a d-c amplifier channel of a feedback control system. In all cases of 
this sort it is desirable that the control system be insensitive to these 
extraneous inputs yet follow the desired input signals properly. 

U (Extraneous input) 






Fig. 7. Block diagram of general feedback control system showing extraneous input. 

Figure 7 shows a block diagram of a feedback control system with an 
extraneous input. The normal closed loop transfer function to the input 
signal is 

C = Oj(8)G a (8) 

R {S) l + G^G^s)' 

and the gain and frequency characteristic of G\(s) G 2 (s) could be established 
as indicated in the previous paragraphs. The transfer function on the 
extraneous signal is 

C G 2 (s) 1 C 

— (s) = = (s). 

U 1 + <?i(s)G 2 (s) Gi(s)fl 



Clearly, the influence of U on C can be reduced by decreasing G 2 (s) and 
increasing Gi(s) (keeping the product G\(s) G 2 (s) constant), and this 
procedure has to be followed until C(s)/U(s) is reduced to acceptable 
values. This may be reasonably easy to do if the u disturbance is limited 
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to a certain frequency bandwidth, so that G 2 (s) need be reduced only in a 
limited frequency region. Otherwise it may be necessary to make fairly 
basic changes in the control system to reduce G 2 (s) and increase Gi(s). 
For instance, although an attempt is usually made in good servomechanism 
design to choose motors and gear ratios to load to minimize moment of 
inertia, it may be necessary to choose gear ratio and motor to increase 
greatly the moment of inertia and thus reduce the susceptibility of a system 
to load torque disturbances. 

In some instances, the extraneous inputs may dictate that certain types 
of systems or components cannot be used. An example of this is the 
stabilized platform used in long-range inertial navigation systems. The 
platform is maintained horizontal for short time periods by reference to 
gyros mounted on the platform. As the vehicle carrying the platform 
moves, this motion is measured by accelerometers mounted on the plat- 
form, and this information is used to precess the gyros (and therefore the 
platform) to keep the platform horizontal as the vehicle moves around the 
earth. The accelerometers also sense errors in the horizontal alignment 
of the table by reference to gravity. The system is shown in the simplified 
one-channel block diagram of Fig. 8. Angular motions of the vehicle 
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Fig. 8. Block diagram of one channel of inertial navigation system showing influence 
of tachometer stabilization on extraneous input of vehicle angular motion. 



during the journey are an extraneous input which influences the gimbal 
servo design. Normally, a tachometer, geared to the shaft of a driving 
motor, is an acceptable way to stabilize a servo. In this case, however, 
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vehicle angular motion signals are amplified by such a tachometer channel 
and the stabilizing has to be accomplished by a rate measurement of 
platform motion alone such as can be obtained by a rate gyro. 

Synthesis of Feedback Control System with Pole- Zero Location 
Techniques (Ref. 3) 

Synthesis of the signal channel amplification and stabilizing charac- 
teristics can also proceed from an orderly tracing of the influence of poles 
and zeros of transfer functions, both of the closed loop and of the open 
loop. In this procedure : 

1. System characteristics are first translated into terms of closed loop 
transfer function poles and zeros. 

2. By a combined graphical and numerical process, the open loop transfer 
function poles and zeros are determined from the closed loop poles and 
zeros. 

3. The known power element transfer function can then be divided into 
the open loop transfer function to obtain series stabilizing components. 
A modification of this last step allows the synthesis of internal feedback 
transfer functions. 

This procedure has an advantage over the frequency response approach 
in that system requirements are more easily and directly translated into 
the pole-zero values than frequency functions. This is particularly true 
of requirements concerning transient response. On the other hand, the 
procedure can easily lead to physically unrealizable, or at least physically 
impractical, transfer functions unless the proper arrangement of closed 
loop poles and zeros is chosen. Thus, the method in practice usually is 
not the direct synthesis approach that it at first appears. 

The method has the disadvantage that the work involved increases 
rapidly with the complexity of the system which is dictated largely by the 
power element. 

Determination of Closed Loop Poles and Zeros from System Re- 
quirements. The closed loop characteristics of a feedback control system 
are usually dominated by a pair of complex conjugate poles so that it is 
customary to express system performance in terms of these dominant 
poles with corrections for the influence of other poles and zeros. In the 
following, performance characteristics will be given in terms of a system 
which contains only two poles and then of more complex systems. Many 
of the transient characteristics associated with closed loop pole-zero con- 
figuration have already been discussed in Chap. 22, and they will be only 
listed here. Derivation of the relationships of Table 1 are given in 
Ref. 3. 
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Table 1. Relation between System Characteristics and Closed Loop 
Poles for System Containing Only Two Complex Conjugate Poles 



V = -faoiiVl - f 2 
System Characteristics Symbol 

Time to first peak of transient response t p 
to a unit step function 

Magnitude of the first peak of a tran- M p 
sient response to a unit step 

Settling time (to 2% of final value) of t s 
a transient response to a unit step 

Number of oscillations until settling N 
time 

Bandwidth of frequency response (mag- a>& 
nitude down 3 db) 

Dynamic velocity error coefficient K v 

Dynamic acceleration error coefficient K a 



Magnitude 

7T 



Vi-r 2 



COO 



exp 



^ 



(~vr=^) 



+ i 



4 



Wo 



2-v/i - r 2 or ts_ 

Vl - 2f 2 + V2 - 4f 2 + 4f 4 



coo 
2T 
coo 2 



4f 2 



The quantities of Table 1 are modified by the presence of additional 
poles and zeros in the closed loop transfer function, as shown in Table 2. 

If only the two dominant poles are involved, it is easy to convert system 
requirements of error coefficients, bandwidth or transient response into 



Dominant pole 



\. 




Additional pole j0*> /$* v° 




41 



.Zero 




H «i *- 



Real axis 



Fig. 9. Angle and distance relations between dominant pole and additional zeros and 

poles. 
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Table 2. Relations between System Characteristics and Closed Loop 

Poles for System Containing Two Complex Conjugate Poles Plus 

Additional Poles and Zeros 

Characteristic Magnitude ° 

u, exp{-- 71 %rn^irn^l} + i 

I Vl-rU PqPdJ L q Z q J J 

Uh w °V / l-2f 2 +)o )0 2 A 2 ) + 'V / 2^if 2 +2(coo 2 A 2 )+4r 4 -4f 2 (a;o 2 /2 2 )+(conV2 4 ) 
JL ^- S I +2 ± 

K v COO Zq Vq 

K a K* v, 2 Zq 2 

t S) N t s and N are substantially influenced by additional poles and 

zeros, and it is usually not practical to handle these quan- 
tities in this way. 

"2^3 is the sum of the angles from the additional closed loop real axis zeros to the 
dominant pole and 2^4 is the sum of the angles from the real axis poles to the dominant 
pole. 

q VqVd 

is the product of the ratios of the distances from origin to the poles, to the distances 
from the poles to the dominant pole, and 



n 



ZqVd 



is the product of the ratios of the distances from the zeros to the dominant pole, to the 
distances from the origin to the zeros. See Fig. 9. (These expressions are somewhat 
approximate, but the error is only a few per cent for practical and useful pole-zero con- 
figurations.) 

co and f . If additional poles and zeros are involved, the transient response 
and bandwidth requirements still largely determine co and f and the 
additional zeros and poles must be chosen to give the required error coef- 
ficients, by use of the proper relations listed in Table 2. Additional poles 
are not inserted because of any system improvement they give but only 
because of physical necessity. The closed loop transfer function must 
contain as many poles as the power element transfer function since it is 
physically impossible to create isolated zeros in the stabilizing transfer 
function to cancel out poles in the power element transfer function. The 
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best that can be done is to select closed loop poles with large negative 
values so that their influence on the various design requirements is small. 
The value of these poles should be 3co or greater to reduce their impor- 
tance. Zeros (with a companion pole of large magnitude) may be added 
purposely to the closed loop transfer function to increase the value of the 
error coefficients. Zeros also increase the bandwidth but not nearly as 
much as the error coefficients. This is shown in Fig. 10. Dipoles are very 
useful for increasing the error coefficients. A dipole is composed of a zero 
and a pole of nearly the same magnitude. The influence of a dipole on 
transient response and bandwidth may be very small and yet if both pole 
and zero have small values, the increase in the error coefficients may be 
considerable. This may be shown by a simple example. For a system 
with a pair of complex poles coo = 10, f = 0.5, K v = 10, co& = 13.6, 
t p = 0.3624 and M p = 1.163. Now add a dipole with p = 1, z = 0.9524. 
The co , tp, and M p values are changed to 12.72, 1.3628, and 1.149 respec- 
tively, but K v is increased to 20. 

Determination of Open Loop Poles and Zeros from the Closed 
Loop Poles and Zeros. The open loop transfer function 

C(s) N(s) 



E(s) D(s) 
is desired. The closed loop transfer 

C(s) N(s) N(s)/D(s) N(s) 



R(s) D r (s) 1 + N(s)/D(s) D(s) + N(s) 

is synthesized from requirements. 

Clearly the open loop zeros are the same as the closed loop zeros. The 
open loop poles are indicated by the expression, D r (s) — N(s). This can 
be solved by a straightforward procedure forming the D r (s) polynomial 
in s by multiplying the (s + pi)(s + p 2 ), etc., factors together where 
Pi, p 2 , etc., are the closed loop poles, similarly forming the N(s) polynomial 
from the zeros, performing the indicated subtraction algebraically, and then 
factoring the resulting expression for the open loop poles. 

The last two steps of this procedure may be replaced by a graphical 
process in which D r (s) and N(s) are plotted as a function of the real variable, 
s = cr, for negative values. The two plotted functions intersect at values 
of s, which are equal to the poles of D(s). This graphical method works 
out easily only if a proper choice of closed loop poles and zeros is made. 
First of all, except for the dominant complex pair, all the poles should be 
real. Then the plot of D r (s) crosses the axis at the values of these poles, 
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and the plotting is somewhat simplified. Secondly, there should be only- 
one zero so that N(s) can be plotted as a straight line. Typical examples 
are shown in Figs. 11 and 12. 
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Fig. 11. Example showing graphical solution for D(s) from plots of D r (s) and N(s). 

N(s) = 2000 (no zeros), 
D r (s) = s 3 + 30s 2 + 200s + 2000 

- (s + 25.2) (s + 2.4 + j8.55)(s + 2.4 - J8.55), 
D(s) = s(s + 10)(s + 20). 



Once the open loop poles and zeros have been determined, and therefore 
the open loop transfer function, the power element transfer function may 
be divided into it thus obtaining the required added transfer function. 
That is, if the required open loop transfer function were found to be 
N(s)/D(s) and the power element transfer function were N p (s)/D p (s), then 
the added or series compensating transfer function would be N(s)/D(s) X 
D p (s)/N p (s). 

In some cases, stabilization by internal feedback may be desired. If 
N(s)/D r (s) is the required closed loop transfer function and N p (s)/D p (s) 
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Fig. 12. Example showing graphical solution for D(s) from plots of D r (s) and N(s). 

\ N(s) = 8s + 3.2, 

D r (s) = s 4 + 6.2s 3 + 9.2s 2 + 9.6s + 3.2 

= (s + 4.627)(s + 0.497)(s + 0.538 ± jl.044), 
D(s) = s(8 + 2)(s + 2)(s + 4). 

is the power element transfer function, then the feedback transfer function, 

N h (s) D r (s) D p (8) 

H(s) = = 1 - 



D h (s) N(s) 



N P (s) 



The above equation can be written in terms of the open loop transfer 
function N(s)/D(s) as 

N h (s) = D(s) D p (s) 

D h (s) N(s) N p (s) ' 
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The feedback transfer function can be derived in a straightforward 
manner from either of the above equations. A graphical procedure similar 
to that used for the series stabilized case is not feasible except for very- 
simple examples. 

2. COMPENSATING COMPONENTS: D-C SYSTEMS 

After the power element, with known log magnitude diagram, has been 
chosen, the task remains to add control components to the system so that 
the system log magnitude diagram has the required characteristics, as 
described in Sect. 1. There are two general ways of adding control com- 
ponents to achieve this purpose : 

1. Add series stabilizing and control components in the signal channels 
which control the power element. 

2. Add internal feedback control loops around the power element. 

Series Compensation 

Series components may be phase lead, phase lag or a combination. In 
Figs. 13, 14, and 15, a sample log magnitude diagram for a power element 
is shown. This sample is typical of an electric motor and many other power 
elements. This power element is compensated by the three different series 
components mentioned above. 

Phase Lead Compensation. Figure 13 shows the log magnitude dia- 
gram for a phase lead component having the transfer function 

ilf K(l + T lS ) 

-^r( s ) = 



E (1 + T 2 s) 

2T(l+i«/«i) 



in frequency function form. 



(1 -f- jtaM 

K is a constant of magnitude that gives the required static error coefficient 
when multiplied by the gain term in the power element, coi is chosen so 
as to make crossover occur within a long section of — 20 db per decade 
slope. This insures adequate phase margin at crossover. Figure 13 also 
shows the resulting system log magnitude diagram. The phase margin at 
crossover may be increased or decreased by adjusting the co 2 /coi ratio. 
Phase Lag Compensation. Figure 14 shows the log magnitude dia- 
gram for a phase lag component having the transfer function 

ilf KQ. + T 2 s) 

— (s) = 

# (1 + T lS ) 

K(l 4- jco/ W2 ) 

= in frequency function form. 

(1 4- ju/wi) 
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Fig. 13. Phase lead compensation. 

The frequencies coi and co 2 must be chosen so that the gain of the system is 
reduced at a low enough frequency to cause crossover in the —20 db per 
decade region of the power element. Figure 14 shows the resultant log 
magnitude diagram. 
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Lag-Lead Compensation. Figure 15 shows the log magnitude dia- 
gram for a lag-lead component having the transfer function 

E S (1 + T lS )(l + T±s) (l+juMil+ja/vt) ' 

Figure 15 also shows the resultant system log magnitude diagram. 
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Fig. 15. Lag-lead compensation. 



In the examples given above, the simplest components were used. These 
had log magnitude slopes of +20 or -20 db per decade. Other slopes 
and characteristics are frequently required and are available in networks 
as given in a later section. 

Comparison of Networks. For a given static error coefficient, the 
phase lead compensation gives the highest crossover frequency. This 
may make the system susceptible to noise in the signal channel. The 
phase lag compensation gives the lowest crossover frequency and tends 
to make the system sluggish and have large errors during transients. The 
lag-lead compensation gives a situation which is intermediate between these 
two extremes. By proper adjustment of time constants, considerable flex- 
ibility of system characteristics may be obtained. 

The lead network has an inherent attenuation at low frequencies which 
has to be balanced by additional amplifier gain to achieve a desired error 
coefficient. In such a system there is an added danger of amplifier satura- 
tion during synchronizing transients. Saturation can lead to instability 
unless the lead network is located ahead of the saturating amplifier in the 
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error channel. On the other hand, a lag network reduces the chance of 
amplifier saturation, but if saturation does occur in the error channel 
beyond the lag network, instability may result. 

In type two, or higher, systems, a lag network cannot be used. 

Feedback Compensation 

This method of compensation uses the principle that the closed loop 
characteristic is nearly equal to the inverse of the feedback characteristic 
whenever the open loop gain is much larger than one. In this way, the 
desired open loop log magnitude characteristics can be obtained by adding 
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internal feedback components with log magnitude characteristics equal 
to the inverse of the desired open loop characteristic. This principle can 
be demonstrated by the equation of the inner closed loop (see Fig. 16). 



G 2 (s) 



G 2 (s)H(s) 



C 

— («)=- 

M (1 + G 2 {s)H(s)) H(s) 1 + G 2 (s)H(s) 

Whenever G 2 (s)H(s) is much larger than unity, C/M approaches 1/H(s). 
Figure 16 shows the equivalent block diagram transformation. The feed- 
back compensation may be in a variety of forms : 

Rate Feedback. Many different types of rate measuring components 
for feedback control systems are available. Commonly used types are: 

1. Tachometers, which are mechanically connected to the power element 
of angular position control systems. 

2. Stabilizing transformers, which effectively measure the rate of change 
of voltage in regulating systems. 

3. Rate gyros, which measure the rate of change of airplane heading in 
automatic pilot systems. 

Figure 17 shows the log magnitude diagram of the same power element 
used in the section on series stabilization. It also shows the log magnitude 
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Fig. 18. Internal rate plus lead network feedback. 
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and phase diagram for an internal rate feedback element, the closed internal 
loop and the log magnitude diagram of the outside open loop. The gain 
of the internal loop must be greater than unity at frequencies up to and 
somewhat beyond the desired crossover frequency of the outside loop and 
the gain, K\, may be set to give the proper crossover. 

Much of Fig. 17 can be constructed by using approximate straight line 
diagrams, but portions of the diagram in the frequency region near cross- 
over of the internal loop should be corrected by using accurate magnitude 
and phase values from the log magnitude-angle diagram (Nichols chart, 
see Chap. 21). 

Rate and Lead Network Feedback. In some systems, it is necessary 
to obtain higher gain at the low frequencies. This can be obtained in a 
system using internal feedback by adding a lead network to the rate feed- 
back. Figure 18 shows the log magnitude and phase diagram of such an 
internal feedback element along with the system diagram leading to the 
open loop diagram of C/E(s). 

For the higher order lead networks, the inner loop may be unstable by 
itself. In such a situation it is necessary to determine the number of 



Imaginary 




Fig. 19. Nyquist diagram of unstable inner loop as indicated by two clockwise rotations. 

positive real poles in the closed inner loop in order to apply a stability 
criterion to the outer loop. An example of such a system is shown in Figs. 
19 and 20. The closed inner loop contains two positive real poles as in- 
dicated by the two encirclements of the — 1 point by the Nyquist sketch 
of the open inner loop transfer function. For the outer loop, and there- 
fore the whole system, to be stable, the Nyquist plot must encircle the 
— 1 point twice counterclockwise. It does this as indicated in Fig. 20. 
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Fig. 20. Nyquist diagram of outside loop. Stability is indicated by two counter- 
clockwise rotations about —1. 

Lead Network Feedback. A lead network is a rate measuring device 
that is somewhat inferior in performance to the components mentioned in 
the paragraph above, but because of simplicity and low cost is often used 
in place of these more expensive components. Such a network is equivalent 
to a tachometer, or other rate device, at low frequencies, but it does not 
have rate characteristics above a frequency which is equal to l/T where T 
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Fig. 21. Comparison of tachometer and lead network characteristics. 
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is the time constant of the network. Figure 21 shows a comparison of 
tachometer and network characteristics. The tachometer also has a high- 
frequency droop in its log magnitude diagram, but this is usually well 
above any frequencies of interest in the feedback control system. The 
lead network can also have a rate characteristic out to high frequencies 
by reducing the network time constant, but this lowers the gain of the 
circuit at the frequencies of interest. Such a lead network feedback is 
particularly useful in systems in which a d-c voltage is one of the inter- 
mediate outputs. An example is the voltage rate feedback around the 
amplidyne in a voltage regulator. 

Multiloop Systems. In the preceding discussion, internal feedback 
loops have been closed to form a portion of the open outside loop of a 
feedback control system. In the same way this second, or outside, loop 
can be closed by using the log magnitude-angle diagram, and becomes a 
portion of a third feedback control loop. This procedure can be extended 
to any number of concentric feedback loops, such as may be present in 
a complex feedback control system. However the block diagram of com- 
plex control systems often is not in the form of concentric loops. Chapter 
20 shows how intertwined block diagrams can usually be transformed into 
concentric loops by making use of superposition rules. 

Alternate Methods of Representation 

The preceding paragraphs have shown how the log magnitude diagram 
of a power element can be modified by the addition of series or feedback 
components to obtain the log magnitude diagrams synthesized in Sec. 1. 
There are several over forms in which these same data may be presented 
and handled to obtain the same desired results. Some of the more com- 
monly used forms are the Nyquist diagram, the inverse complex plane 
diagram, and the root locus plot. 

Nyquist Diagram and Inverse Complex Plane Diagram. The use 
of these diagrams is described in detail in Chap. 9 of Ref. 1. Since these 
diagrams contain exactly the same information as the log magnitude dia- 
gram, essentially the same principles as described above may be used. 
The steps may be summarized: 

1. Select the starting axis (type of system) and gain factor from the 
static error coefficient requirements. 

2. From stability and transient response requirements, determine the 
maximum allowable M and draw in this M circle. 

3. By using the gains established in step 1 and the chosen power element, 
draw a Nyquist diagram. 

4. Add frequency sensitive networks (or proper internal feedback loops) 
as needed to reshape the diagram to avoid the required M contour. See 
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Fig. 22. (This is a trial and error process which will become more efficient 
with the user's experience.) 



Nyquist diagram of 

power element using 

required gain 




Imaginary 



Nyquist diagram of 
compensating network 



Re 




System Nyquist diagram 



Power element diagram 

modified to meet 
stability requirements 



Starting axis and 

low-frequency 

gain established 

by requirements 



Fig. 22. Nyquist diagram showing synthesis procedure. 



The Nyquist diagram is used in this discussion because of its historical 
position, although it is somewhat easier to use the inverse complex plane 
plot in this type presentation. 

Root Locus Plots. (See Ref. 6.) This is essentially a complex plane 
graphical representation of the pole-zero configuration synthesis presented 
in Sect. 1. In this plot, the locations of the closed loop poles are traced on 
the complex plane as the open loop gain is varied. Use of this diagram 
may be broadly outlined in steps analogous to those of the preceding para- 
graph : 
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1. Select the closed loop poles from system specifications of performance 
and stability. These may be located on the complex plane. 

2. Start with the poles and zeros of the power element and draw the 
root locus plot. 

3. Add open loop pole and zero combinations to modify the root locus 
plot to pass through the required closed loop poles. Reference 6 indicates 
optimum selections of added pole and zero configurations to achieve the 
desired changes in locus shape. 

Figure 23 indicates the above steps. 
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Fig. 23. Root locus plots showing synthesis procedure. 



Comparison of Alternate Methods of Representation with the 
Log Magnitude Diagram. The log magnitude and phase diagrams con- 
tain the same information as the Nyquist or inverse complex plane diagram. 
Because the log magnitude diagram is much easier to construct than the 
other two diagrams, and required modifications to meet specifications are 
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more easily visualized and constructed on the log magnitude diagram, 
there is normally no reason for using the Nyquist or inverse diagram in 
control system design work. In very complex systems, containing several 
positive real roots, it may be desirable to make rough order of magnitude 
Nyquist sketches to check rotations about the — 1 point in order to check 
stability but the actual numerical work should be done using log magni- 
tude diagrams. 

The log magnitude diagram is also easier to construct than the root 
locus plot and would normally be used for problems concerned with stabi- 
lity, bandwidth, static and low-frequency errors. However, the root locus 
plot has more specific information regarding actual transient response 
characteristics and damping factor, and would be used in problems in 
which this type of information is of primary importance. 

Design Aids 

Charts of Electric Networks. Tables 3 through 7 show many of the 
electric networks useful in compensating d-c systems. All these networks 
are of the resistor and capacitor type since any practical type of frequency 
characteristic can be obtained with these components. Inductance is 
also a useful circuit component, but large time constants cannot be ob- 
tained in sizes competitive with resistance-capacitance components. 



Table 3. Stabilizing Networks: Lead Networks with 20 db/Decade Slope 

(Ref. 2) 



. Network 



Attenuation Characteristic 



AC 




Mdb/dec 



Co- 



G. - 1 



-CZD 1(- 










20*/dec^_ 
V i 

^^ i 




db 


<5„ 




1 l 


<0 



r« 



AC 



NR 




(c) 



Y^ 



-CZh 

NR 



C > 



(<*) 




1^ 



£7? AC 






20 db/deci 






B(£ + AT) 



£« AC 



j Dfl 



G 



NR 



If) 




B B(E + N) 



ER AC 

-czi — \i~x 



NR ^_ 

H=] 1 



x 



I 1_ 
r, r, 



■20db/dec 



<?. 



B(E + G + V) f CAT 



(B + NHE + G) + BN 



(g) 



23-30 



Table 3. Stabilizing Networks (Continued) 
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Table 4. Stabilizing Networks: Lead Networks with 40 db/Decade Slope 

(Ref. 2) 



Attenuation Characteristic 
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Table 4. Stabilizing Networks (Continued) 



Transfer Function 
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Table 5. Stabilizing Networks: Lag Networks with 20 db/Decade Slope 

(Ref . 2) 



Attenuation Characteristic 
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Table 5. Stabilizing Networks (Continued) 



Transfer Function 
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Table 6. Stabilizing Networks: Lag Networks with 40 db/Decade Slope 

(Ref. 2) 



Attenuation Characteristic 
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Table 6. Stabilizing Networks (Continued) 



Transfer Function 
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Table 7. Stabilizing Networks: Lead-Lag Networks (Ref. 2) 
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Table 7. Stabilizing Networks (Continued) 



Transfer Function 
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Table 7. Stabilizing Networks (Continued) 
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Table 7. Stabilizing Networks (Continued) 



Transfer Function 


r, 


r, 


T,T,t' + Tn + 1 


D + N " RC 


A(D + N)RC 


T.3V' + [r, (l + £\ + T,l « + 1 


T ' T '\'+B,n°l + m]" + ' T ' + Ti "* 1 


BHRC 


A(D + E + N)RC 


*»[' + w¥SftsJ' + K' + S) + *J' + 1 


r ' T '[ , + Mi> *Um]' + ! , '' + '''[ 1 + «. + ffl<£ + tf + «]." M 


TTg» rc 


MD + E + N)RC 




?••'■■[' + M+ "J + m]'+| 1- ' + ' , -[> + ( , + 0l M + I + ml|' + ' 


irh" RC 


A(.D + E + N)RC 


+H-+i)+?o>+'+¥)+»[-+,.+s£:?+ w ] 



23-41 



Table 8. Mechanical Components, Lead 
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Table 9. Mechanical Components, Lag 
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Table 10. Mechanical Components, Lag-Lead 
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Table 11. Mechanical -Hydraulic Components 
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Symbols in Table 12 

P c = controlled variable-pressure signal 
P r = referenced input 
Pf = feedback pressure 
P m = output pressure 

Po = equilibrium value of P m and Pf when P c = P r 
Ai, Az = differential areas acted on by pressures as shown in diagram 
A<i = area of feedback diaphragm 
(R = variable resistance in line 
Q = capacitance associated with Pf 

Charts of Mechanical Networks. The great majority of feedback 
control systems use electrical signals in the error channel, but in some cases 
it is necessary or desirable to use mechanical devices to handle the signals. 
The signals are most commonly in the form of linear or angular positions. 
Tables 8 through 10 show a few mechanical networks for obtaining phase 
lead and phase lag characteristics, for linear motion. Components for 
rotational motion can be readily derived from these. Oil-filled dashpot 
elements indicated in these networks have a disadvantage that their 
characteristics change with extreme variations of temperatures so that 
these networks may not be applicable in some systems. Several more 
mechanical networks are given in Ref . 7. 

Charts of Mechanical-Hydraulic Networks. Effective stabilized 
networks can also be formed from hydraulic system components. Many 
hydraulic systems contain one or more stages of hydraulic amplification 
consisting of control valve and output piston components. When mechani- 
cal feedback around these stages is used, lead or lag characteristics can 
be obtained as indicated schematically in Table 11. 

Chart of Mechanical-Pneumatic Networks. Pneumatic systems 
allow considerable flexibility in generating control functions. Table 12 
shows several typical circuits. Additional circuits are given in Ref. 8. 

3. COMPENSATING NETWORKS: A-C SYSTEMS 

Many of the components that are commonly used are alternating-current 
devices. Selsyns are accurate and reliable error measuring devices; a-c 
amplifiers are relatively simple and drift-free devices compared with, their 
d-c counterparts ; a-c tachometers have reached a high state of development 
for small, low-power servos; and the two-phase a-c motor is a popular drive 
for low-power, fast servos. Much analysis using a modulated a-c carrier 
is the same as that used in d-c systems, but there are additional design 
problems as described in the following sections. 

Requirements for Ideal Series Networks. Frequently a-c servo- 
mechanisms are required to operate at frequencies above that indicated 
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by the motor time constant. In such systems a rate producing component 
is required to stabilize the control system. For a series stabilizing com- 
ponent, this usually takes the form of an electric lead network. 

The special problem of an a-c rate network is that the network must 
take the derivative of the envelope of the carrier without shifting the 
phase of the carrier. Consider a carrier of time function cos w c t modulated 
by the signal m cos au c t, where a is a fraction much less than 1. (The 
useful control frequencies in an a-c feedback control system must be 
quite low relative to the carrier frequency.) If this signal is operated on 
by a lead network of transfer function 1 -f- Ts, where T is the lead time 
constant, the required output signal is 

m[cos au} c t cos oi c t — Td(d c sin aco c t cos o) c t] 

= m cos a> c £[cos ao> c t + au c T cos (au c t + 90°)]. 

Figure 24 is a vector diagram of these signals, showing the phase lead 
produced in the modulating signal. Note that the phase of the carrier is 



/ 
/ 



/ 



aco c T cos (au c t + 90°) 



cos aw c t 

Fig. 24. Phase lead produced by ideal lead network on modulating signal in all a-c 

servo system. 

not changed although the phase of the modulating signal is. The equation 
can be changed to the equivalent expression 

m 

— {[cos (1 -f- a)u c t + cos (1 — a)o} c t] 

+ a« c T[cos ((1 + a)u c t + 90°) cos ((1 - a)o> c t - 90°)]}. 

This equation indicates that the modulated carrier signal is actually 
composed of two frequencies somewhat displaced about the carrier. 
Furthermore, it indicates the necessary frequency response of a network 
to obtain an effective lead characteristic at the modulation frequency, 
au c . Figure 25 shows the magnitude and phase characteristic indicated 
by this equation for several values of T. 

Practical Networks. The ideal frequency characteristic shown in 
Fig. 25 cannot be attained with physical networks. However, the fre- 
quency response can be approximated over part of the frequency rang 
by several practical networks. The most common of these are the bridged 
T and parallel-T networks. (See Table 7(a).) The bridged-T networks 
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have characteristics similar to the ideal network of Fig. 25 whereas the 
parallel-T has the rate portion only. Addition of a parallel channel to 
the parallel-T network makes it possible to approximate the lead charac- 
teristics of Fig. 25. 
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Fig. 25. Characteristics of ideal lead networks for use in a-c feedback control systems. 
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As indicated in Table 7(a) the transfer function of the bridged-T net- 
work is 

e_o T x T 2 s 2 + T 2 s + 1 

et " TiTV 2 + {[1 + {N/D)]T l + T 2 }s + l' 

where 

/ DN \ 

T l = i— )HRC, 

\D + N/ 

T 2 = A(D + N)RC. 

The circuit is adjusted until T\T 2 = l/co c 2 , where co c is the carrier fre- 
quency. The minimum value of the transfer function occurs at co = co c , 
where the value is 

1 1 



g 1 + [1 + (tf/Z>)](!ZVT 2 ) 
The above equation can be rewritten 



e 



(s 2 /V) + Vts/t&m + i 



e { (s 2 /co c 2 ) + VTjTi g(s/u c ) + 1 

Characteristics of this network for several values of T 2 /Ti are shown 
in Fig. 26. It is seen that this can approximate the ideal characteristic of 
Fig. 25. The factor T 2 /T\ largely influences the rate of change of angle 
near co c whereas g determines the magnitude of phase change that can be 
obtained. 

Characteristics of the parallel-T network can be found in Ref . 5. 

Sensitivity to Carrier Frequency Shift. The most serious weakness 
of a-c feedback control systems using series networks is that the system 
performance is impaired by normal shifts in the frequency of the carrier. 
For example, in an aircraft power system, the 400-cps power may be fre- 
quency regulated to only 5 or 10 per cent. In many a-c feedback control 
systems using series stabilizing networks, this amount of frequency shift 
will render the system completely useless. 

Figure 27 shows the effects of a carrier frequency shift on the operation 
of an a-c stabilizing network. These are: 

1. The gain of the control systems at low frequencies is increased. This 
may result in saturation in subsequent elements in the control system. 

2. The phase of the carrier is shifted. This means that any phase sensi- 
tive devices such as discriminators or a-c motors will not operate at best 
efficiency. 
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Fig. 27. Effect of carrier frequency shift on the operation of an a-c series stabilizing 

network. 



3. The phase lead at the control system frequencies (aw c ) is decreased. 
Thus the network does not perform the function for which it was intended 
and instability may result. 

It is noticed that "fast" control systems, which are designed to have a 
high crossover frequency (large values of a) are less susceptible to carrier 
frequency shifts than systems which have a large effective lead network 
time constant. 

Tachometer Stabilization. Alternating-current tachometers are ex- 
cited by the carrier frequency along with other components in an a-c 
control system and generate an amplitude-modulated signal proportional 
to velocity. This operation is not hindered by reasonable changes of the 
carrier frequency so that this method of stabilization is not subject to the 
limitations of a-c stabilizing networks. The analysis or synthesis of a-c 
systems using tachometers can proceed just as in the case of the d-c system 
described in Sect. 2 under Rate Feedback. 

Other Techniques. The previous sections have discussed means of 
stabilizing all a-c feedback control systems by using rate-producing com- 
ponents. Often it is necessary to add gain at low frequencies and reduce 
this gain below the desired crossover frequency by means of an integrating 
or reset component. This is the type of system (using d-c signals) described 
in Sect. 2, Phase Lag Compensation. 
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Theoretically, an a-c carrier lag network can be constructed by using a 
bridged-T circuit in the feedback channel of a feedback amplifier. How- 
ever, since the time constant of a lag network has to be considerably larger 
than used in a lead network, the effect of carrier frequency shifts usually 
makes this method impractical. 

Commonly lag networks are obtained by rectifying the carrier to direct 
current and then using d-c networks. This procedure then is the same as 
that of Sect. 2 and the system is no longer an all a-c system. 

Another method to obtain an effective lag network is to use a small 
"reset" servo in parallel with the signal channel, as shown in Fig. 28. 
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Fig. 28. Reset servo channel in parallel with signal channel. 

The servo channel has high gain at low frequencies but, because of the 
tachometer feedback, this gain falls below that of the regular signal channel 
below the crossover frequency. 

4. OPEN-CLOSED LOOP CONTROL 

Open-closed loop control, sometimes called schedule and trim, is not so 
much a different kind of control as it is a different way of visualizing or 
synthesizing a control system. The principle is that an open loop control 
system, although not accurate enough for the complete control, responds 
predictably and stably to an input signal, and it can be used as an ap- 
proximate, or first order correction, control system. Then the required 
accuracy can be obtained as a correction or "trim" to the open loop and 
is accomplished by the use of a relatively slow but high gain feedback 
loop. This is illustrated in the block diagram of Fig. 29, which is modified 
for analysis purposes in Fig. 30. The only closed loop to consider is the 
easily stabilized, low crossover frequency loop and that the required high- 
frequency response is obtained by the open-ended forcing function. In 
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the actual system this forcing action is attained by the parallel signal 
channel to the power element. 

An allied situation exists when there is difficulty in measuring the control 
system output accurately and immediately. The trouble may be in an 
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Power element 



Fig. 29. Block diagram of open-closed loop control. 

inherent delay in the measuring device, such as the time lag of a thermo- 
couple measuring temperature, or it may be caused by the need for a 
smoothing and averaging process to attain accuracy from noisy data. In 
such cases, the delayed but accurate measurement can be used in a trim 
feedback control loop and then an internal, fast response, feedback loop 
is formed by using an alternate measurement. This alternate quantity is 
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Fig. 30. Modification of Fig. 29 for analysis. 

related to the desired output in a known way, such as a pressure change 
which accompanies a change in temperature, but accuracy of such a re- 
lationship is not high enough to use the alternate quantity as the ultimate 
measurement of the desired output. 

Open-closed loop control is covered in detail in Ref . 9. 
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1. INTRODUCTION 

Linear systems can be designed to obtain a desired response to commands 
and disturbances which may be exactly defined either by an equation or 
by a graphical plot (Chaps. 19 through 23). In many cases inputs can 
be described adequately only in a statistical manner. Examples are the 
jitter observed in automatic radar tracking systems and gust disturbances 
to an aircraft. This chapter covers methods for: 

(a) Measuring and describing statistical inputs. 

(6) Computing the system response to such inputs. 

(c) Specifying optimum designs. 
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2. MATHEMATICAL DESCRIPTION OF NOISE 

Random processes are described in Chap. 12, Sect. 16, and Chap. 13, 
Sect. 2. It is sufficient to note here that a random process has a complete 
set of probability distribution functions. If these distributions are in- 
dependent of time, the process is stationary and its characteristics can be 
defined by time averages (Ref. 1). 

Autocorrelation. The most useful description of a random process for 
control system analysis is the autocorrelation junction <f> defined by eq. (1) 
for a stationary function of time x(t): 



(1) 



1 r +T 

<t>xx{r) = lim — I x(t)x(t + t) dt. 



Figure 1 graphically illustrates eq. (1). For nonstationary processes the 
autocorrelation may be described by an ensemble average that is a function 



x(t + T) 




Fig. 1. Illustration of the computation of the autocorrelation function. 
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of time as well as r. This definition is given in eq. (2) : 

(2) <f> xx (t, t) = I I XiX 2 P(t, x lf t + T, x 2 ) dx u dx 2 , 

J —x J — 00 

where P(t, x\, t + r, a; 2 ) is the joint probability density of X\ at time t and 
x 2 at time (£ + t). Important 'properties of the autocorrelation function for 
stationary series are: 

(3) <Mr) ^ <M0), 

(4) 0xx(r) = c/> xx ( — r), 

(5) <f> xx (0) = ?. 

In eq. (5), the bar indicates a time average. 
An interesting example is the autocorrelation of the function sin cot. 

x(t) = A sin cot. 

(6) 4>xx(j) = lim — I A sin a>£ sin oo(t + t) d2 

r->«> 27 1 J_r 

= A 2 /2 cos cot. 

Although sin cot is not strictly stationary, this example illustrates the effect 
of a pronounced periodicity in noise data. If it exists, it will show up in 
the autocorrelation as cosine function. 

Cross-Correlation. In some cases a control system will have two in- 
puts, x and y, which are not completely independent. The relationship is 
expressed by the cross-correlation function defined by eq. (7) for stationary 
series : 

i r +T 

(7) 4> xy (r) = lim — I x(t)y(t + r) dt. 

T->oc 21 J _t 

For nonstationary series <j> xy must be expressed as an ensemble average as 
given by eq. (8) : 

I xyP(x, t,y,t + t) dx dy. 

—oo •'— oo 

Important properties of the cross-correlation function for stationary series are: 

(9) fo*v( T )] ma x < x * or y 2 (whichever is larger), 

(10) <$>xy(r) - 0i/x(— t). 
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Fig. 2. (a) and (6) Examples of autocorrelation and spectral density pairs. 

The autocorrelation of the sum of two correlated functions is given by 
eq.(ll): 



(11) 



4>X+y{r) = $ XX (T) + (j>yy(T) + 4> X y{T) + <j>y X (T). 



If x(t) and y(t) are independent: 

(a) The cross-correlations become constants equal to the products of 
their means or xy. 

(b) The autocorrelation of the sum becomes the sum of the individual 
autocorrelations plus twice the product of the means, or </> xx (r) + (fryyir) 
+ 2xy. 

Spectral Density. An alternate description of a stationary random 
process is the spectral density, $(co). It is a measure of the distribution of 
energy in the frequency spectrum. For a voltage wave the units would 
be volts 2 per radian per second. 

The following discussion is not rigorous but will show the 'physical 
significance of $(w). Assume that several samples of noise of duration T 
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* fa) 
2a0/ir(a 2 + 2 ) H 




(c) 




$(0))=P 



<j>(t) = 2ttP8(0) f 



Fig. 2. (c) and (d) Examples of autocorrelation and spectral density pairs. 

seconds have been expanded in a Fourier series of the form shown in eq. 
(12): 

(12) x(t) °° 

where u n = 2nir/T, 

2 



+ 23 c n COS (fa n t + ypn), 



2 r* 

a n = — I x(t) cos oi n t dt, 
T J 

2 r T 

b n — — I x(t) sin o) n t dt, 
T Jq 



c n = a n + &„ , tn = tan (6„/a„). 

Assume that T is very long compared to the longest periodicity present 
in the function. If the c's and i^'s for a given value of n are considered 
over a large number of samples, it will be found that the ^'s are uniformly 
distributed between -\-ir and — it and that the c n 2 's have an average value 
c n 2 . A knowledge of c n 2, s for a given process is sufficient to predict the 



24-06 FEEDBACK CONTROL 

output of a linear control system with a transfer function G(jcc) between 
the noise input and the system output. The mean square of the output is 

c 2 co 

(13) , <r c 2 = |G(0)| 2 -^+£c 2 |G(« B )l 2 . 

2 w=i 

The experimental determination of the c n 2 's would be relatively inefficient. 
However, the c n 2 's are related to the spectral density by eq. (14) for large T. 

(14) c? = 2$(co n )Aco, 

where 

co n = nAco = 2tt7i/T. 

Hence if the average value of the input is zero, c is zero and eq. (13) 
becomes eq. (15) : 

(15) a c 2 = 2 f $(«) I G(to) | 2 da>. 

Jo 

The spectral density is related to the autocorrelation function by the 
Fourier cosine transform as shown in eqs. (16) and (17) : 

1 r +co 

(16) $(0?) = — I 0(r) COS cot dr, 

2ir •/_ oo 

(17) $(r) = f $(co) cos cor dco. 

The cross-spectral density & X y{<*) bears the same transform relationship 
to the cross-correlation function <f> xv (T) as the spectral density does to the 
autocorrelation function. 

Figure 2 shows several pairs of spectral density functions and auto- 
correlation functions. 

3. MEASUREMENT OF NOISE 

The greatest problem involved in the analysis of the response of a control 
system to a random input is obtaining the required characteristics of the 
input. If the input is stationary, long samples are necessary and numerous 
calculations must be made. In most cases the use of high-speed digital 
computers is necessary if the job is to be completed in reasonable time. If 
the input is nonstationary, the magnitude is multiplied many times since 
the results must be calculated separately for each value of time. 

Calculation of <£(r) for Stationary Inputs. The most straightfor- 
ward methods of analysis if the input is stationary are to compute the auto- 
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correlation function defined by eq. (1). The approximate form for cal- 
culation is given by eq. (18) : 

-i N—tn 

(18) 0(m At) = — — - 22 x n x n+m , 

N - m -f- 1 n=0 

where At is the time interval at which values of the function are read and 
x n is the value of the function n At seconds from the beginning of the 
sample. 

Sampling Rate. The value of At is set by Shannon's sampling theorem 

(19) A,-±. 

The value of / s is the highest frequency present in the data. In general, 
this will not be known. In most control systems there are considerations 
other than noise which set an upper bound on the system band pass so 
that a filter may be inserted in the device which records the sample to 
eliminate frequencies not of interest. This is desirable since by increasing 
At the number of calculations is reduced. 

Required Range of t. The maximum value of t is determined by the 
use to which #(t) is to be put. If it is desired to compute the variance of 
a system output, reference to eq. (29) will show that m max At should equal 
the longest anticipated settling time of the system output to an impulse 
applied at the noise input. 

An insight to the effect of using a finite value, m max , can be obtained by 
performing the integration of eq. (16) over a finite range or by multiplying 
the true autocorrelation function by a function u(t) which equals unity 
for — T < t < + T and zero elsewhere. Then the approximate spectrum 
is given by eq. (20) : 

1 r +co 

(20) ^approxM = — I u{j)4>(t) COS COT dr. 

2tt J_ ot 

Since the true spectrum is the Fourier transform of <£(t), and (T/ir) 
(sin a)T/o)T) is the transform of u(t), the approximate spectrum is given 
by eq. (21): 

T r + 00 sinaT 

(21) $approx(co) = — I *(&> — a) — da. 

7T •/— oo aT 

For example, if the time function were a pure sine wave of frequency co 
of unity power, the true spectrum would be given by eq. (22) : 

(22) *(«) = i[«(coo) + *(-«o)], 
where 5(w) = impulse at co. 
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Then if the autocorrelation were computed only for (— T < r < -\-T), 
the approximate spectrum computed from the result would be given by 
eq. (23): 



(23) 



$„ 



<(«) = 



T 



2tt L (w - oi)T 
Figure 3 shows a plot of these results. 



sin («o — oi)T sin (co + od)T' 



(co + co)r 




Fig. 3. 



Effect of limiting the range of r on computation of the spectral density: (a) 
exact spectrum; (6) approximate spectrum. 



From this example, it can be seen that the usable range of (t) limits the 
resolutions of the measurement. Also, a limited range of (r) may lead to 
negative values of <i>(co) which are physically impossible. 

Required Sample Length. The required sample length for computing 
$(t) depends on the use to which the result will be put and to a certain 
extent upon the frequencies contained in the input function. A useful rule 
of thumb is that N in eq. (18) should be at least 10 times the maximum value 
of m. As a check, the autocorrelation can be computed for two samples 
of equal length. If then the results are nearly equal, the samples are 
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probably long enough. If not, the average of the two should be compared 
with results from a sample twice as long and so forth until agreement is 
reached. 

Once an approximate knowledge of the frequency components of the 
noise has been obtained, a better estimate can be made of the required 
sample length. Reference to eq. (28) shows that computing the system 
output from the autocorrelation function of the input is equivalent to 
computing the output directly by convolution and averaging the square. 
Hence, if a sample of the system output T seconds long is sufficient to 
give an accurate measure of the output mean square, a sample T + T s 
seconds long, where T s is the system settling time, is sufficiently long to 
compute the autocorrelation of the input. If <£o( r ) is the estimated output 
correlation, the ratio of standard deviation of the computed output mean 
squares o- 2 ^] for samples T seconds long to the true mean square is given 
byeq. (24): 



(24) 



g ko 2 ] 
* 2 o 



LTV(0) Jo 



f (T- 



t)<£o 2 (t) dr 



Figure 4 shows a plot of this ratio where <}>o(t) is (e q|t| ) asa function of 
(a.T). References 2, 3, 4 give a more detailed consideration to the problem. 



= ^VaT-l+e- 2 "?)* 




Fia. 4. Standard deviation of errors in computing the output mean square error from 

a finite length sample. 



Nonstationary Inputs. Any process observed in nature is nonstation- 
ary in the strict sense of the term. However, in many cases the input 
characteristics will vary so slowly that samples long enough to compute 
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#(t) can be considered stationary. In this case the techniques previously 
discussed are applicable. 

A slightly more difficult problem exists when the input changes too rapidly 
to obtain a sufficiently long sample, but it still does not change appreciably 
during the settling time of the control system. In this case, several record- 
ings of the input must be obtained over the range of characteristics of 
interest. Then short samples can be drawn from common points and the 
autocorrelation averaged. The amount of computation required is vastly 
increased. 

One type of slowly changing nonstationary function can be treated in 
a more simple manner. If the frequency components retain the same 
amplitudes relative to each other but where absolute magnitude increases 
or decreases with time, a single recording can be used. The input is divided 
into several short samples. The autocorrelations of each sample are nor- 
malized, so that the function is divided by <£(0), and the normalized func- 
tions are averaged. The autocorrelation for any specific time is then the 
averaged normalized autocorrelation function times the mean square value 
of the input corresponding to that time. Note that this technique is 
helpful only if it is known that the spectrums have the same form. Other- 
wise, more data would have to be taken to establish the point. 

If the input characteristics vary appreciably during a settling time of 
system, the problem becomes immensely complicated. To compute 
4>{t, t\) it is necessary to average the products from many recordings of the 
input. At least one hundred products would be necessary to obtain 10 per 
cent accuracy when the output of a control system is calculated. 

Correlation Computers. Special computers for the computation of 
correlation functions can be built where many correlations must be done 
and high-speed digital computers are not available. The basic principle 
is illustrated in Fig. 5. The noise is recorded on a media such as magnetic 



Q 



®9 9® 



© 



_ d 



Tape speed 



Multiplier 

Integrator 

Fig. 5. Correlation computer. 



tape and played back through two reading heads spaced a distance d 
apart. The second reading head receives the same signal as the first except 
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that it is delayed by a time equal to d divided by the tape speed. The 
two outputs are then miltiplied and the result integrated. The output of 
the integrator is then given by eq. (25) : 



(25) I =ff(t)f(t ~ 



r)dt. 



I may then be divided by the time interval over which the run is made to 
get the autocorrelation. Other possible recording media are photographic 
film and ink recordings tracked by hand (Ref. 5). Still other methods use 
pulse sampling and storage. 

Computers of this type depend on the availability of the noise in the 
right form. For instance, radar tracking data are usually taken with a 
moving picture camera so that the data must be read frame by frame before 
they are useful. Also, several records may have to be combined to arrive 
at the noise. In such cases, it would be simpler to use a general purpose 
digital computer. 

The accuracy of such systems is limited by the recording mechanism, 
multiplier, and integrator. To obtain reasonable accuracy these units 
become bulky and expensive. 

4. SYSTEM RESPONSE TO NOISE 

Time Domain Methods. The response of a linear dynamic system to 
an input x(t) is given by the integral in eq. (26): 

(26) c(t) = J x(t - t)o(t) dr. 



it) = f x{t - 
Jo 



Substituting this eq. (26) in eq. (1) gives the autocorrelation function of 
the output if the input is stationary: 



,+T „oo 



(27) cc (r) = lim — I I g(s) I g(r)x(t + r - r)x(t - s) dt ds dr 

t—ko 2i t/_7' Jq Jo 

= I g(s) I g(r)<j> xx (T + s - r) ds dr. 
Jq J o 

The mean square value of the output is obtained by setting r = in eq. 
(27): 

(28) c 2 (t) = I g(s) ) g(r)4> xx (s - r) dsdr. 
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By an appropriate change of variables the alternate form of eq. (28) is 
given by eqs. (29) and (30): 

,+00 

(29) 6 2 {t) = | <Mt)0„(t) dr, 



(30) 



'CO =f_ 

«« = r 

*J 



g(t)g(t + r) dt. 



Strictly speaking the transition from eq. (28) to eq. (29) is possible 
only if the input x(t) is stationary. However, if the characteristics of 
x(t) vary only slightly during one settling time of the control system and 
if 0(t) has been computed from a sample which is long compared to one 
settling time, eq. (28) is approximately true for nonstationary inputs. 

For the more general case of a linear time-varying system with a non- 
stationary input, the mean square output at time (t) is given by eq. (31): 

(31) c 2 (t) = I g(t, s) I g{t, r)<f>(t -r,r- s) dsdr. 

Jo Jo 

The autocorrelation function is defined in this case by eq. (2). The func- 
tion g(i, s) is defined as the effect on the system output at time t of an 
impulse applied at time (t — s). The wavy line over c 2 (t) in the equation 
indicates an ensemble average rather than a time average. 

Frequency Domain Methods. For cases where the spectral density 
is known, the mean square of the output is given by eq. (32) : 



(32) 



c 2 (0 =2f *(a)\G(j<*)\ 2 d<*. 
Jq 




*-m 



x(t) 



y(t) 



G^G^} 



l+GiG 2 



l + G a G 2 



o — ><w 



(b) 
Fig. 6. Control system with two inputs: (a) actual circuit; (6) equivalent circuit. 
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A more general case is shown in Fig. 6 where there are two inputs to a 
control system and the inputs may be correlated. In terms of the equiv- 
alent circuit, the mean square output is given by eq. (33) : 



(33) 



c\t) = 2 f [ | G x (ju) | 2 * sx («) + | G v (jo>) \ 2 ^M 
Jo 

+ G x *(ja)Gy(j<a)$ xy (u) + G x (ja)G y *(ja)$ vx (a>)] da. 

The starred transfer functions are complex conjugates. 

Computer Methods. A modification of eq. (28) leads to an analog 
computer method for computing noise output. If the input noise has a 
constant spectral density K, the autocorrelation function becomes a delta 
function at r = with strength 2ttK. Then the system output is given 
byeq. (34) T 

(34) ~cH() = 2tvK f g 2 {r) dr 



S 



G n 




Control 
system 




Squaring 
circuit 




Integrator 



Generating the impulse response by analog techniques, squaring it, and 
integrating the result give the mean square output. If the input noise 

JL 

Impulse 

Fig. 7. Computer simulation to compute the mean square output for a correlated noise 

input. 

does not have a constant spectral density, the output can be computed 
from the system shown in Fig. 7. The filter transfer function is specified 
byeq. (35): 

(35) G n (ja)G n *(jo>) = $„(«). 

The same technique can be used for linear time-varying systems, and 
for nonstationary random inputs if the inputs are equivalent to stationary 
noise passing through a linear time- varying filter. For this case eq. (34) 
becomes eq. (36) : 

(36) c 2 {t) = 2vK I g 2 (t, r) dr, 

where g(t, t) has the definition given following eq. (31). The function 
g(t, t) with t as the variable can be generated from the adjoint of the 
control system and a shaping filter (Ref. 6). The following is quoted from 
Ref. 7. "The adjoint is found from the analog of the original by: 

1. Turning each element in the loop around and reversing the direction 
of signal flow. 
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2. Letting the variation of time- varying element start from some time 
ti and run backward relative to the action in the original system. 

3. Interchange the input and output of the system. The new input is 

Kt - h)." 

The output is then g(t, t) as a function of r. This output can then be 
squared and integrated to give a machine solution of eq. (36). Figure 8 
shows an example of a system (a) and its adjoint (6). 

f x (t) 

i 



T 



Multiplier 



l + T lS 



1 + T 2 s 



K 



->c(t) 




Impulse T 



(a) 



1 + Txs 



1 + T 2 s 



fo(t) 



Multiplier 



-> g(t v t) 



(b) 
Fig. 8. Analog of a system (a) and its adjoint (6). 

Noise Generators. Many nonlinear control systems and systems in- 
volving a human operator will not yield readily to analytical techniques. 
In these cases simulation using a random noise generator is required. 
Several such generators (Refs. 8, 9, 10) are available. In general, the 
output is a flat spectrum with various amplitude distributions possible. 
Where the spectrum of the true input noise is known, a shaping filter, as 
specified in eq. (35), can be used to modify the output of the noise generator. 

Although the use of a noise generator and a simulated system provides 
a simple solution to many complex problems, it is also a time-consuming 
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one. The methods of the previous section of this chapter provide an 
indication of the sample lengths required where non-time-varying systems 
are tested with stationary inputs. For time-varying systems, the answers 
will be of interest at one or more times during the run. The number of 
runs required is determined by standard statistical methods. 

5. SYSTEM DESIGN IN THE PRESENCE OF NOISE 

Previous sections of this chapter have shown methods for describing a 
random input or disturbance and methods for computing the response of 
a system to these inputs. It remains then to establish procedures which 
can be used to apply these methods to the design of control systems. 
Unfortunately each problem is a little or greatly different from any general 
case so that the designer must examine his problem and determine what 
methods are adequate. 

Mean Square Error Criteria. Practically all the work covered in this 
section aims at minimizing the mean square error of the system. In some 
cases restraints are placed upon the solution in an attempt to conform 
more nearly to the practical situation. The limitations of this approach 
are listed below. 

1. The mean square error may not be the proper criteria. For instance, 
in a gun fire control system the object is to maximize the probability of 
destroying the target. 

2. The data concerning the system inputs will seldom be exact enough 
to warrant an extended analysis or to justify the system complexity re- 
quired to realize the desired response. 

3. The optimum design may be very sensitive to practical limitations of 
the system, such as gain variations. 

As a result, it is suggested that formal methods of optimum design are 
good guides for a design but that more useful results are obtained by 
starting with a conventional design and varying the parameters to mini- 
mize the noise error as computed by formulas in Sect. 3 of this chapter 
or by analog computers. 

Optimum Design for Stationary Random Inputs. For the case 
where the signal and the noise enter the system at the same point and are 
both random and stationary and their cross-correlation is zero, the linear 
filter giving the least mean square error is given by eq. (37) (Ref. 11): 

(37) G op t(j«) = „,,,„,, G d {jo>), 

where $ s (co) = signal spectral density, 
*n(w) = noise spectral density, 
Gd(jui) = desired transfer function if noise was not present. 
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The mean square error is given by eq. (38) : 

•+* $ s (c)$ n (co) 



(38) 



<T 



2 



emin 



r— $ s (c)$ n (co) 

^/ x ■ ^ / x I ^0«) I 2 dto. 

J-n $ s (co) + $n(C0) 



Unfortunately, G op t(jw) will usually be physically unrealizable since 
its impulse response will have values for t < 0. TTie 6es£ 'physically real- 
izable filter is derived from eq. (37) as follows. Consider a filter made up 
of two filters in cascade as in Fig. 9. The input spectral density 



Signal 

+ 
noise 



Filtered output 



Fig. 9. Optimum filter broken into two series filters. 

[$/(«) = $ s (co) -f- $ n (w)] is split into two functions $/ + (w) and <l>/ _ (co) 
where $ + contains all the poles and zeros in the upper half -plane and $j~ 
contains all the poles and zeros of the lower half-plane. Then Cri(co) is 
given by eq. (39) : 

(39) GiW = l/$/ + (<o); 
for example, if 

(40) $ s (co) = 



u 2 + a 2 

(41) *„(«■>) = tf n 2 . 

Then the input spectral density is 



7f w 2 (co 2 + b 2 ) = K? 

co 2 + a 2 ; K n 2 



(42) *,(«) = - ;" , ■ 9 ' ; 6 2 = — + a 2 , 

and 

(43) *<*> -(££)©• 

The transfer function of G 2 (joi) is obtained by taking the physically 
realizable part of the remaining part of the optimum transfer function, 
[$ s (co)(rd(jco)/$j~~]. This is obtained by finding the impulse response of 
this remaining part and constructing a filter that will give the same result 
for positive values of time. For instance, if the desired response to the 
signal is unity, the term in brackets becomes, for our example : 

*.(a>)G d 0«) K* 

(44) 



$ 7 (co) K n (b - ju) (a + ju) 
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For positive values of time, the impulse response of this filter is 

K 2 e~ at 



(45) 



g + (t) 



K n (a + b) 



The transfer function of a realizable filter which has this impulse response 
is obtained by taking the Fourier transform of eq. (45). 



(40) 



K 2 ( 1 \ 1 

\a + 0/ a -f- 



K n \a -\- 0/ a -\- ja> 
The complete filter is then the cascade of Gx(ju>) and G 2 (ju>), or simply 

K s 2 /K n 2 y 



(47) 



Giiiju) = 



(a + 6)(6+ico) 



By referring to eq. (42) it is seen that as the noise amplitude becomes 
very small, b becomes very large and the gain of the filter becomes unity, 
and that as the noise becomes very large, the gain goes to zero and gives 
an intuitive check of our results. If Gr 12 0'co) is known, conventional methods 
may be used to synthesize a control system approximating the optimum 
response. 

In some applications a lag can be tolerated in the output or conversely 
a lead is desired. In these cases Gd(ju) is no longer unity, but it is given 
by e JuT where T, the time displacement, is plus for a lead and negative for 
a lag. Figure 10 shows a plot of the impulse response of G(jco) without 





(a) (b) 

Fig. 10. Effect on the optimum response of allowing a lag T: (a) zero lag; (6) lag equals T. 




Fig. 11. Physically realizable part of 10(6). 

regard for physical realizability for both zero and negative values of T. 
For the case of the lag the realizable transfer function for G(jco) would 
have the impulse response shown in Fig. 11. This response is physically 
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realizable but would require a very complex network. The advantage of 
accepting the signal delayed by T seconds is that it would be more accurate 
when it arrived. 

The same results can be obtained by writing an equation for the optimum 
impulse response of the control system. This is known as the Weiner- 
Hofi equation: 

(48) <f> ss (r) = I *jj(t)0(t) d 



= I 07/0 

Jo 



In eq. (48) S s(t) is the autocorrelation of the signal, 4>n{r) is the auto- 
correlation of the input, and g(r) is the optimum impulse response. The 
Fourier transform of g(t) is the same as Gwiju) derived previously. Equa- 
tion (48) can be solved numerically if the autocorrelations are the result of 
experimental measurement and no analytical expressions for them are 
available. 

Nonstationary Inputs. Equation (48) can be extended to the case of 
nonstationary inputs by rewriting it as follows (Refs. 12 through 15) : 

(49) cj> ss (t, t- t) = \ <f> n (t - r,t- r)g(t - r, t) dr, 



where g(t — r, t) is the response of the control system at time t to an im- 
pulse applied at time (t — r) and the autocorrelation functions are defined 
by eq. (2). Equation (49) will usually require a numerical solution. 

The response g(t — r, t) is the optimum only at time t. A different 
time will in general require a different response so that the system must 
be time-varying. The values of g(t — r, t) are most easily obtained by 
using adjoint techniques with an analog computer (Ref. 7). A cut-and- 
try variation of parameters can be used to approximate the calculated 
optimum response. 

System Optimization Under Constraints. The response called for 
by the minimum squared error criteria may call for unrealistic demands 
on components by requiring extended ranges of operation to prevent 
saturation or by requiring highly rated components to prevent overheating. 
Hence, the optimization should more practically be carried out under the 
restraints of rms power dissipated in the output or signal level at various 
points in the system (Refs. 16 and 17). 

Example. Suppose that the input consists solely of a signal with spec- 
tral density <£/(co) and it is desired to limit the rms velocity of the output. 
The mean square error and velocity are given by eqs. (50) and (51) where 
G(joo) is the system transfer function. 
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(50) a t 2 = 2f ^(co) | 1 - G(jco) | 2 rfco, 

Jo 



dc{t) I 2 
(51) 



dt I 
The quantity / is formed: 

dc{t) I 2 



= 2f w 2 $/(co)|G(jw)| 2 d«. 
Jo 



dt 



(52) I = a/ + X 

(53) = f (*/(«) 1 1 - G{ju) | + Xco 2 $ 7 (co) | (?(j w ) | 2 ) do>. 

Jo 

The object is to minimize I by the proper choice of G(ju>). The result is 
a function of X the Lagrangian multiplier. With this result, the value of 
(dc(t)/dt) 2 is plotted versus X and a value of X is picked that gives the 
minimum error for the velocity below the specified limit. The solution 
of eq. (53) is quickly obtained in this case by noting that / would be the 
mean square error if \co 2 <J?j(co) were a noise input. Hence, the optimum 
transfer function without regard to realizability is 

$/(co) 

(54) G opt (ico) = 



$/(«) + Xco 2 $j(a>) 

which has the same form as eq. (37). Hence, the methods of treating 
eq. (37) can be used in this case. 

Summary. The designer should be careful not to expect too much of 
formal techniques of optimum design. On account of shortcomings in 
the mean square error criteria and the difficulty of obtaining accurate 
statistical description of the inputs, they will seldom produce practical 
designs. They do, however, form useful guide posts and establish limits 
on performance. 
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1. DEFINITIONS 

Definition of Nonlinear System. A nonlinear system or element is a 
system or element described by a nonlinear differential equation. The 
significant feature of a nonlinear differential equation is that the principle 
of superposition is not applicable. Therefore, in a nonlinear system the 
simultaneous signal level and operating range of all inputs must be specified 
in order to define the system performance. 

Classes of Nonlinearity : Denned by Static Characteristic (Refs. 1 
and 2). An essential nonlinearity is a nonlinearity which is necessary to 
the basic operation of the system. A relay servo is a good example. By 
its definition, this type of nonlinearity must be included in any system 
analysis. It is also referred to as an intentional nonlinearity. 
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Parasitic nonlinearities are those nonlinearities which are small or in- 
cidental defects in an otherwise linear system. This type of nonlinearity 
is also referred to as an incidental nonlinearity. Neglecting parasitic non- 
linearities often does not compromise the early analysis and the preliminary 
designs can be based on this assumption. However, in more complex, 
high performance systems and/or systems which are required to follow 
large step inputs, these nonlinearities may result in unsatisfactory or even 
unstable operation. These nonlinear effects must, therefore, be considered 
in a complete analysis. 
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Fig. 1. Components of a simple system. 

Examples. Typical sources of parasitic or incidental nonlinearities in a 
simple system, Fig. 1, are: 

(a) Preamplifier and/or power amplifier saturation. 

(b) Preamplifier, power amplifier, or motor sensitivity limit (due to 
Coulomb friction or stiction). 

(c) Hysteresis in motor characteristics. 

(d) Motor velocity and acceleration limits (due to magnetic saturation). 

(e) Inherent nonlinear motor characteristics which are linearized for 
analysis, e.g., series or 2-phase motor characteristics. 

(/) Granularity in the followup potentiometer. 
(g) Backlash and stiction in the gearing. 

Classes of Nonlinearity : Denned by Dynamic Effect. A slowly vary- 
ing nonlinearity is a nonlinearity which changes system characteristics 
slowly relative to the response time of the system. Such nonlinearities 
are often related to the variation of an independent parameter outside the 
control system. The variation of the effectiveness of the control surfaces 
of a missile with altitude is typical of this type of nonlinearity. 

A rapidly varying nonlinearity is a nonlinearity which changes system 
characteristics at an appreciable rate with respect to the response time of 
the system. Such nonlinearities are usually functions of the dependent 
variable, e.g., amplifier saturation. 
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2. GENERAL NONLINEAR SYSTEM PROBLEM 

Importance of Considering Nonlinearities. Conceivably a system 
could be designed to operate in an essentially linear manner for all expected 
inputs. Such a design would probably not be economically feasible. The 
ratings of power elements, amplifiers, and precision of gear trains, etc., 
required for such a design would be prohibitive from space, weight, and 
cost considerations. Fortunately such a design is usually not required 
and may not be optimum even if achieved. Normally smooth operation 
is required only for small deviations about the operating point. For large 
deviations, saturation of amplifiers and/or power elements is often accept- 
able. The problem for large deviations is to assure that the change in 
system operating characteristics does not result in unsatisfactory perform- 
ance when synchronizing on the new operating point. 

For small deviations, there exists the problem of minimizing the non- 
linearities which cause deadband or signal distortion. Again it is not 
usually practical to eliminate the cause. The designer attempts to reduce 
the effect to acceptable limits. The threshold at which an error signal 
will cause control action is initially a static problem. However, those 
nonlinearities which are important at low signal levels can cause a self- 
sustained oscillation of low amplitude. It is necessary to assure that the 
amplitude and frequency of these oscillations are not sufficient to affect 
adversely system performance. 

Of great importance also is the fact that linear operation represents only 
a small cross-section of the possible methods of achieving the desired 
operation. It certainly is to be anticipated that in many cases the best 
performance can be achieved by nonlinear operation. In fact in a number 
of cases techniques using nonlinear elements have been employed to im- 
prove the performance of servos without other improvements in existing 
controllers and motors. (See Sect. 7.) 

State of Art in Nonlinear System Study. There are two types of 
tasks in nonlinear studies, analysis and synthesis. Practical analytical 
techniques of treating nonlinear systems are not available except for the 
most rudimentary and simple systems. Fortunately, the task of analysis 
lends itself to solution by computers. The analog and digital computer 
no longer makes it necessary to rely upon tedious hand methods of cal- 
culation. Although the ennui of calculating performance of nonlinear 
systems has been removed, the problem of organizing a general theory 
still remains. The continuing advancement of any scientific field is de- 
pendent upon the ability of the persons involved to generalize, classify, 
and catalog their results. This provides the foundation for future develop- 
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Table 1. Common Nonlinear Phenomena 



Type Condition for Occurrence 

1. Jump Occurs in systems with light- 

reso- ly damped resonances, and 

nance nonlinear restoring forces 

that are excited sinusoid- 
ally. Describing function 
analysis shows that 
| G | cos y > 1 
must be true for phenom- 
ena to occur in single loop 
system with simple satura- 
tion. 

(\G\ = open loop gain, 
7 = phase margin). 
(See Ref. 3.) 



Characteristics and/or Effects 
When excited by a sinusoidal driving sig- 
nal, the resonance is normal for small 
amplitude signals. Theoretically, for 
larger amplitude signals the resonance 
bends as in A below. In practice the 
three- valued function cannot be meas- 
ured but the response will appear to 
jump as in B below. The jump will 
occur at different values of frequency 
depending upon whether the frequency 
is increasing or decreasing. The phase 
characteristic exhibits a corresponding 
jump. 



Decreasing 
frequency jump 
l\C Increasing 

frequency 
{ V^s^Jump 



Frequency- 




General Remarks 
For jump resonance to occur the system 
must be second order or higher. To 
have significant bending the damping 
must be 0.1 or less in a second order 
system. The phenomena can occur in 
systems with saturation or increasing 
gain characteristics. The bending is 
to the right for increasing gain charac- 
teristics and to the left for saturation. 
The normalized second order equation 
of the type dx 2 /dt 2 +■ dx/dt + f(x) = 
F cos cot (Dufnng's equation) has been 
solved for various forms of the func- 
tion f(x) and each case has exhibited 
the jump resonance when the viscous 
damping was small. (See Ref. 4.) 
The existence of the jump resonance 
can be confirmed by the use of describ- 
ing functions. (See Refs. 3 and 5.) 
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Frequency- 



oscilla- 
tions 



3. Subhar- 
monic 
genera- 
tion 



tion analysis shows that 
conditions of | G \ = 1 and 
7 = must be met for phe- 
nomena to occur in simple 
systems. For a stable sys- 
tem with an unstable limit 
cycle it is necessary to ex- 
cite the system beyond the 
level of the limit cycle to 
obtain self-sustained oscil- 
lations. 



Appears in nonlinear systems 
excited sinusoidally. No 
general rules are available 
defining the necessary con- 
ditions for occurrence. The 
phenomena have been ob- 
served in lightly damped 
systems with nonlinear re- 
storing force and in systems 
with nonlinear energy de- 
lays. 



can oscillate at fixed amplitudes. Such 
oscillations are referred to as limit cycle 
oscillations. Limit cycles can be either 
stable or unstable depending upon 
whether the oscillation converges or 
diverges from the conditions repre- 
sented. Depending upon the system 
characteristics, the limit cycle oscilla- 
tion can vary from nearly simple har- 
monic oscillation to a highly nonlinear, 
relaxation type oscillation. Self-ex- 
cited oscillations arising in a stable 
system with unstable limit cycle are 
referred to as soft oscillations. Self- 
sustained oscillations which occur aft- 
er the system has been excited to a 
given level (unstable limit cycle) are 
referred to as hard oscillations. 



When the output contains subharmonics 
of the input exciting frequency, the 
phenomena is referred to as svbhar- 
monic generation. 



uration will contain both a stable and 
unstable limit cycle and an unstable 
system with saturation will have one 
stable limit cycle. System imperfec- 
tions that appear at low signal levels 
(backlash, friction, etc.) can, under 
the proper conditions, cause limit 
cycle oscillations. Existence of this 
type of limit cycle makes it necessary 
to define instability in terms of the ac- 
ceptable magnitude of an oscillation 
since a low level nonlinear oscillation 
may or may not be detrimental to per- 
formance of the system. Because soft 
and hard types of oscillations can exist, 
the designer must specify the input 
range completely in evaluation or syn- 
thesis of a nonlinear system. Limit 
cycles can be most correctly explained 
by use of the phase plane; however, 
the magnitude and fundamental fre- 
quency of the limit cycle can be esti- 
mated to a first order of magnitude by 
means of describing functions. 

Systems with elements having hystere- 
sis, i.e., backlash, magnetic hysteresis, 
friction, have been known to exhibit 
this type of performance when excited 
with a sinusoidal input. The transi- 
tion from harmonic to subharmonic 
operation can be quite sudden, but 
once the subharmonic is established, 
it is often quite stable. (See Ref. 8 
and its bibliography.) 
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Table 1. Common Nonlineae Phenomena (Continued) 



Type Condition for Occurrence 

4. Inter- Occurs in amplitude-sensi- 

modula- tive nonlinear systems ex- 

tion ef- cited by two or more fre- 

fect on quencies. The frequencies 

gain can be separate inputs or 

one input with a complex 

waveform. The amplitude 

of the complex wave must 

be sufficient to enter the 

nonlinear region. 



Characteristics and/or Effects 
Because of the amplitude-sensitive non- 
linearity the frequencies will be inter- 
modulated. This causes the original 
frequency components to have differ- 
ent amplitudes and phase shift than 
obtained from the nonlinear system 
with only one frequency present. This 
can be interpreted as a different phase 
shift and/or attenuation through an 
element. The effect is also apparent 
when noise is present with the signal. 



General Remarks 



In a simple saturating system the effect 
can be explained quite easily. Two 
frequencies are considered. After the 
saturation the amplitude of both will 
be reduced beyond that expected if 
only one frequency had been present. 
If we are considering the effective gain 
with respect to one of the two frequen- 
cies, the gain will have been reduced. 
This gain reduction in the open loop 
can be interpreted as reducing the gain 
crossover and therefore increasing the 
phase shift of the closed loop for the 
frequency being considered. (See Ref . 
7.) By considering one of the frequen- 
cies as an extraneous signal, the effect 
of noise on the performance of a satu- 
rating system can be envisioned. The 
effect is particularly significant if the 
amplitude or phase shift of the closed 
loop is important to system perform- 
ance. 
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ment. A great deal of work of this type remains to be done in nonlinear 
system analysis. 

The major problem for the systems engineer lies in synthesis of a control. 
In synthesis one needs in addition to a full appreciation of the charac- 
teristics and a complete understanding of the nature of the task to be 
performed: (a) methods of rapidly, approximately estimating the effect of 
different types of compensation in order to allow selection of a potentially 
good approach and (&) having selected a general approach, a logical design 
procedure which converges on the "best" design. Although no such 
generally satisfactory method exists, the methods of linearization, describ- 
ing functions, and phase plane analysis are powerful analytical tools for 
attacking nonlinear problems. (See Sects. 3, 4, and 5.) By and large 
these methods are not exact but often suffice for preliminary design cal- 
culations. The majority of these methods attempt to linearize the problem 
sufficiently to allow the use of the well-known techniques used in the study of 
linear systems. Because of the difficulty in providing generalized design 
criteria or design charts for any but the simplest nonlinear system, the 
synthesis of a system using nonlinear elements is primarily a cut-and-try 
process tempered with common sense. 

Unusual Phenomena Peculiar to Nonlinear Systems. Many un- 
usual phenomena occur in nonlinear systems. In a linear system the 
response to a given input defines the response to be expected from any 
input. This is not true for a nonlinear system. Cases arise where per- 
formance may completely deteriorate between a step response and a sinus- 
oidal response. Table 1 summarizes some of the more common types of 
nonlinear phenomena which have been catalogued. The list demonstrates 
that the designer must be aware of the peculiar characteristics exhibited 
by nonlinear systems and completely specify the operating conditions in 
order to proceed with an intelligent, efficient design. The types of non- 
linear phenomena described in Table 1 are those most commonly en- 
countered in nonlinear feedback control systems. Other types of nonlinear 
phenomena have been catalogued; frequency entrainment, asynchronous 
excitation and asynchronous quenching, parametric excitation, etc. See 
Ref. 6 and its bibliography for more details on nonlinear phenomena. 

3. METHODS OF ANALYSIS: LINEARIZATION 

Frequency response analysis can be used only when the system is de- 
scribed by linear constant coefficient equations. Certain nonlinear sys- 
tems can be linearized by use of the perturbation theory. The method 
assumes that for very small deviations about the operating point the 
system is linear. The perturbation method determines the coefficients of 
the new linear equation describing the performance of the system. Once 
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reduced to a linear form the usual frequency response techniques can be 
applied. 

Method of Evaluating Linearized Coefficients. A nonlinear func- 
tion of one or more variables can be linearized if the function is analytic. 
(Refs. 6 and 9.) Expand the function in a Taylor series about the operat- 
ing point and neglect all the second order and higher derivatives. One 
thereby considers only the incremental change about a nominal value. 

If the function is f{x\, x 2 , x 3 , • • •, x n ) = /(#;), then the Taylor expansion 
about point ai{x\ = a 1} x 2 = a 2 , x 3 = a 3 , etc.) is 



A df(xi) 
V = f&i) = f(a%) + 2^ 



or 

and 

(1) 



=l dX{ 
d 2 f(X{, x k ) 



i » a a /(*) 

(Xi — at) + - 2^ 



-in n 

2, i = i k=i oXidXk 



2~i dXi 2 
(xi — a { )(x k - a k ) + 



(xi - at)'' 



Jc ?± i, 



xk=ak 



A df(xi) 

y = f(xi) ~ f(ai) + 2-j 



Az/=Z 



i=i dXi 
df(x{) 



=l dXi 



(x { - ai) = y + Ay, 



(xi - a t ) 



for small values of Xi — a { . Equation (1) is a linear relation between 
Ay and xi — ai or Axi, the incremental changes. 

The accuracy of the approximation can be estimated by evaluating the 
next terms in the Taylor series, e.g., the second order terms are 



1 j, d 2 j{x x ) 

2 t - =1 Ox 2 



(Xi - a { y 



1 » ^ d 2 f( Xi , x k ) 

2 j = i k =l dX{dx k 



(xi — cii){xk — a k ), k 9± i. 



xk=ak 
xi=ai 



Alternate Method of Evaluation of Linearized Coefficients. The 

same linearization can be accomplished by substituting ^ -f- Axi,a 2 + Ax 2 , 
etc., for x\, x 2 , etc., in the original function, and neglecting all second order 
and higher terms, i.e., terms containing the product of the type AxiAx 2 - 
Here a x and a 2 are the values at the operating point and Ax x and Ax 2 
are small deviations. 
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Example. Consider the nonlinear function f(x, y) = w = xy. Substi- 
tuting x = xq + &x, V = 2/o + Ay, w = w + Aw 

w + Aw = x y + Ayx + x Ay + Ax Ay. 

If Arr and Ay are small, the term Ax Ay is small and can usually be neglected. 

w + Aw « z 2/o + Az?/o + Ayrc . 

Since w = rr 2/o, ^ ne remaining two terms must be deviations, therefore, 

(2) Aw « Axy Q + Ayx . 

Equation (2) is a linear expression for Aw. For small variations about 
the operating point x , y , eq. (2) describes the performance. The coef- 
ficient y is the gain between Ax and Aw, and x is the gain between Ay 
and Aw. 

The same relationship would have been obtained by using the Taylor 
series expansion. 

Table of Useful Algebraic Approximations for Linearization. 
Table 2 is useful when making the above substitution into nonrational 
equations. The terms in Table 2 were determined by considering the 

Table 2. Useful Algebraic Approximations 
m <3C 1 
Algebraic Expression Approximation Next Term in Series 

1. 1 — m -\-rri 1 

1 + m 

2. (1 + mY l+mn + W(n ~ 1)m ' 



Tfl 

3. e m 1 + m + — 



2 

2 



4. log e (1 + m) m 

5. sin (m) m 



m* 

Y 

m 3 
in 1 

y 



6. cos (in) 1 

7. (1 + mi)(l + m 2 ) 1 + mi + m 2 +mim 2 

series expansion of the closed form. The last column of the table can be, 
used to estimate the accuracy of the approximation. 

To use the table, it is necessary to work the expression into a nondi- 
mensional form. 
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Example. Consider the flow through a variable orifice. Flow, q, is 
given by 

q = CoAVp, 

where A = orifice area, a variable, 

P = pressure drop, a variable, 
Cd = flow coefficient, a constant. 

By substituting the incremental change form 

q + Aq = C d (Ao + AA )Vp + AP. 
Dividing the quantity under the radical sign by P yields 

q + Aq = C d (Ao + AA ) VF Vl + AP/P . 
Using approximation 2 in Table 2 for w = 3^ yields the expression 

9o + Aq « C d (^ + A^ ) V^J ( 1 + -^-Y 

\ 2 Pq/ 

By expanding and neglecting higher order and constant terms this reduces 
to 

Aq - &) AP + (C ^ AA ' 

where the terms in parenthesis are the equivalent gain between AP and 
AA and flow, Aq. 

Graphically Linearizing System Characteristic. The analytical 
expression for the function need not be known. If the graphical relation- 




Fig. 2. Determination of the linearized coefficient from a plot of the function. 
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ship between the variables is known, a linear expression can be obtained 
by considering incremental departures from the nominal values at the 
particular operating condition being considered. For Fig. 2 then 

w + Aw = f(x ) + (slope of function at x , w ) Ax 



and 



8w 

= w -\ 

8x 



Ax 



Aw = 



8w 
8x 



Ax. 



*0 



The method can be extended to functions of more than one variable by- 
obtaining the slopes from the appropriate curves. Note that in dealing with 
functions of more than one variable the slope must be taken so as to be inde- 
pendent of all variables but the ones being considered. In Fig. 3 a function of 



Slope = -^ | 




Slope =; 




(b) 



Fig. 3. Determination of the linearized coefficients from cross-plots of a function of 
two variables: (a) xw plot and (b) yw plot. 

two variables is cross-plotted to obtain the independent slopes from separate 
plots. From these the linear relation for small deviations at values of the 
variable x 2 , 1J2 is 



Aw = 



8w 
8x 



Ax + 



8w 
8y 



Ay. 



X 2 ,V2 



For functions of two variables that are reasonably regular it is usually 
possible to pick the values for calculating the slopes from a single plot 
of the function and avoid the labor of cross-plotting the functions (see 
Ref. 9). 

Use of Linearized Coefficients. The characteristics of the function 
or system are approximated by the linearized coefficients for small varia- 
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Ax 



y 



A + 



Ay 



Fig. 4. Equivalent block diagram for 

the linearized small deviation expression 

for the function w = xy. 



tions from the operating point. Therefore, for small disturbances, only the 
deviation terms need to be considered in determining the stability. The 

"constant" terms defining the oper- 
ating point will remain the same to 
a first order approximation. Under 
these conditions the block diagram 
-*~Aw for the multiplication w = xy is 
shown in Fig. 4. 

For small disturbances, the ap- 
proximation can be substituted into 
the system diagram and the usual 
methods of linear analysis and com- 
pensation used. Note that one is 
now dealing only with the deviations 
and not the total variable. For further applications of this method see 
Sect. 7. 

Limitations. The above approximations are limited to small devia- 
tions from the operating point. The errors get progressively worse as the 
signal level is increased, and considerable care must be exercised when 
dealing with large excursions. The validity of the approximation can be 
checked by evaluating the next terms in the Taylor series or the last 
column of Table 2. 

The perturbation theory is valid only if the derivative of the function exists. 
The method would be of doubtful value in dealing with a relay charac- 
teristic. 

The method is sometimes limited when the operating point is at 0. One 
or more of the variables will then have a steady-state value of 0, and 
terms involving that variable and the deviation can be lost. This can 
lead to an indeterminate or inaccurate solution. For instance, in the 
example of linearizing w = xy if either x or y were 0, the variation of 
Aw with the corresponding value of Ax or Ay is zero. See eq. (2). Al- 
though the error of the approximation is still Ax Ay this term becomes 
significant with respect to the other terms as x and y become small. 
Consider the example of the flow through an orifice. The linearized devia- 
tion expression is repeated here: 

/ C d A \ j— 

Aq = wK/ AP + {Cd ^ o) AA - 

As P — > 0, this expression becomes indeterminate and the analysis 
based on the approximation under these conditions loses significance. A 
linear expression no longer adequately describes these situations. 
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4. METHODS OF ANALYSIS: DESCRIBING FUNCTION 

General 

Definition of Describing Function. When an element is excited by 
a sinusoidal signal, the describing function is the ratio of the fundamental 
of the output signal to the sinusoidal input signal. A describing function 
of an element may be a complex quantity, characterizing both amplitude 
and phase relations between the input and output. The describing func- 
tion may be a function of both signal amplitude and frequency. 

Use of the Describing Function. Within the validity of the basic 
assumptions, the describing function, representing the nonlinear element, 
can be substituted directly into the system equations for the nonlinear 
characteristics. The use of the describing function quasi-linearizes the 
frequency response equations. Since the describing function will be a 
function of amplitude, the system frequency response will be a function 
of both frequency and amplitude. The quasi-linearization of the system 
equations in the frequency domain allows the use of the Nyquist criterion 
to determine stability. 

Although the dependency of the frequency response upon both frequency 
and amplitude complicates the calculations slightly, practical methods are 
available which require little effort beyond that normally required in the 
plotting of a Nyquist diagram. 

Usefulness of Describing Function. The describing function provides 
an approach which allows a solution in the frequency domain. The ability 
to manipulate the system equation in the frequency domain is valuable 
because: (a) frequency response techniques developed for linear system 
are available for synthesis, (6) synthesis and analysis can be handled with 
relative ease, and (c) the technique is not limited to systems with few energy 
storage elements. 

Basic Assumptions of Describing Function Method. If the input 
to the nonlinear element is sinusoidal, then it is assumed that: 

1. The output is periodic and of the same fundamental period as the 
input signal. 

2. Only the fundamental of the output wave need be considered in a 
frequency response analysis. 

3. The nonlinear element is not time-varying. 

4. Only one nonlinear element is considered to exist in the system. 

Assumption 1 implies that no subharmonics are generated. 

If the element used in a system is driven by sinusoidal signal, the output 
of the device by assumption 2 is considered to be sufficiently filtered by 
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the system characteristics so that the signal fed back into the input of 
the nonlinear element is essentially sinusoidal. The degree to which the 
input to the nonlinear element must be "essentially sinusoidal" is de- 
termined by how critical the nonlinearity is to the wave shape of the 
driving signal. 

While the describing function cannot be obtained for a system in which 
the coefficients are time-varying because the output would not reach a 
steady-state periodic solution, the describing function can be obtained 
for an element with characteristics which are dependent on frequency. 
In such a case the describing function will be a function of both amplitude 
and frequency of the signal. 

If a system contains two nonlinearities of major importance, it is still 
possible to get a describing function for the system. Often the easiest 
way to obtain the describing function in this case is to lump the charac- 
teristics of the two nonlinearities and obtain an over-all describing function. 
In general, it is not practical to consider each nonlinearity separately. 

Theory of Describing Functions. Consider the system of Fig. 5. It 
is convenient in describing function analysis to have the nonlinear and 
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Fig. 5. Block diagram of simplified nonlinear system. 



linear elements separated as in the figure. In the following it will be 
assumed that this has been done. 

The output, n, of the nonlinear element is related to the input, m, by 

n = [g D (m)]m. 

If the input is sinusoidal, then, by the assumptions made, the error 
M(jui) must be sinusoidal and of the fundamental frequency. 

The output, N(ju>), can be represented by a Fourier series: 
N(jo>) = NiUu) + iV 2 (j2a>) + N*(j&*) + • ■ ■, 
where Ni(ju) is the first harmonic of the output N(ju>). 
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By definition, the describing function is 

#i0«) 



Ni(ju) 



M(j«) 



\M(j u )/ 



angle 

l M(jco), 



where Gd y (\M\, co) is the describing function as a function of amplitude, 
| M | , and frequency, co. 

Usually for convenience (\M\, co) and the subcript 1 are dropped; 
this leaves Gd as the symbol for describing functions. 

Within the validity of the assumption that the harmonics are sufficiently 
filtered by the linear system elements so that the feedback signal contains 
essentially the fundamental, the harmonics of the output of the nonlinear 
element can be neglected, and the describing function Gd can be used as a 
series element in the frequency response analysis. 

Gd can be determined by conventional Fourier series analysis. (See 
Chap. 14 and Ref. 10.) 

Stability Criterion. For the system of Fig. 5 the frequency response 



is approximated by 



C GdGUu) 

« 0") = 



B 1 + G D G(j<*) 

where Gd = the describing function, 

G(jco) = the frequency sensitive portion of the system. 
For a minimum phase shift system (see Chap. 23) the system will be 
critically stable when the denominator is zero or 

1 + G D G(ju) = 0, 
or 

(3) G(j») = -1/Gd, 
or 

(4) -G D = 1/G(i«). 

When eq. (3) or (4) is satisfied, the system will have a sustained oscillation 
of the amplitude and frequency which satisfies the equation (Ref. 11). 
Limitations and Accuracy of Describing Function Method. There 
are two major disadvantages with the describing function analysis: 

1. There is no convenient method to determine the accuracy. A method 
proposed by Johnson (Ref. 12) becomes laborious for more than the most 
simple systems. 

2. Frequency response analysis allows prediction of the transient 
response. A describing function analysis allows at best a qualitative 
interpretation. These designs can therefore predict stability and frequen- 
cies of oscillation, but are limited to crude rules of thumb in prescribing 
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a given stable response. It must be pointed out that such approximations 
are often all that is justified by the accuracy of other system data available. 

In a wide variety of applications, use of describing functions has allowed 
prediction of the frequency and amplitude of oscillations within 20 per 
cent. It is difficult to generalize, but usually the method will be most 
accurate when Gd and G(ju>) are varying rapidly in the region of inter- 
section (Ref. 12). Erroneous results may be obtained in some cases when 
the intersection is approximately tangential (Ref. 13). 

Because the assumption of sinusoidal input to the nonlinear element 
is not exact, the method works best when Gd is not sensitive to wave 
shape of the driving function. 

Note. The describing function method of analysis is the only practical 
analytical technique for treating nonlinear systems which are higher than sec- 
ond order. 

Describing Function : Methods of Presentation 

Inverted Nyquist Diagram. The equation for sustained oscillation, 

-G D = l/G{jiS), 

indicates that an intersection of the — Gd and 1/G(ju>) loci is a point having 
a given frequency and amplitude at which sustained oscillation can occur. 




G(;co) 



Fig. 6. Inverted Nyquist diagram for a stable nonlinear system showing the stable 
and unstable regions for the describing function locus. 

In a normal inverted Nyquist plot the —1 point should be on the left 
as the l/G(jco) locus is traversed in the direction of increasing frequency 
for a minimum phase shift system. In this case, there is no longer a fixed 
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— 1 point, but the system is stable if the — Gd locus is entirely on the left 
of the l/G(jui) locus. , 

The stability criteria is then as follows: If a locus of all possible values 
of Gd is plotted, then the system will be stable if the —Gd locus does not inter- 
sect the l/(r(jco) locus and Gd locus lies completely on the left-hand side of 
the 1/G(ju) locus when the 1/G(ju) locus is traversed in the direction of in- 
creasing frequency. (Valid for minimum phase functions.) 

A stable system is shown in Fig. 6. 

Note. By plotting in this manner the frequency sensitive, G(joS), and 
amplitude sensitive, Gd, portions of the system have been separated and can 
be considered independently. 

Nyquist Diagram. Equation 3 leads to a Nyquist diagram as indicated 
in Fig. 7. 




Stable region 



Increasing co c G(jco) locus 



Fig. 7. 



Nyquist diagram for a stable nonlinear system showing the stable and unstable 
regions for the describing function locus. 



The stability criteria is as follows: // a locus of all values of —1/Gn is 
plotted, then the system will be stable if the — 1/Gd locus does not intersect 
the G(jca) locus and the 1/Gd locus lies entirely on the left-hand side of the 
G(jo)) locus when the G(jco) locus is traversed in the direction of increasing 
frequency. 

A stable system is shown in Fig. 7. 

Log-Angle Plane Representation. It is sometimes more convenient 
to work with magnitude and phase angle semi-independently. This can 
be done in the case of describing functions by use of the log magnitude- 
angle plot. These are the familiar coordinates used on the Nichols charts. 
(See Chap. 21, Sect. 7). 
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For the case of a nonlinear system, the critical point is 

20 log 10 | (?(;«) | =201ogio|l/G!D| 

(5) ^Gti*>) = -180° - ZG D . 

If the conditions of eqs. (5) are met, the system is unstable. A typical 
plot of a stable system servo is given in Fig. 8. As long as the two loci do 
not intersect, the system will be stable. 
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Fig. 8. Log magnitude-angle diagram for a stable nonlinear system showing the stable 
and unstable regions for the describing function locus. 

Typical Loci for Nonlinear Systems. Table 3 shows a number of 
different types of loci which can be expected. In each of these diagrams, 
the frequency and amplitude loci have been plotted. The arrows indicate 
increasing frequency and amplitude of signal input to the nonlinear 
element. 

System B has an intersection at point 1. This indicates that the sys- 
tem will be unstable at amplitudes less than those at point 1 and will be 
stable for larger amplitudes. This system, therefore, will be unstable and 
oscillate at the amplitude and frequency of point 1. If a small disturbance 
is introduced into the system B described in Table 3, the system will 
appear unstable and the amplitude of oscillation will increase until point 
1 is reached. If the amplitude of oscillation becomes larger than this, 
the system appears stable and any oscillation would tend to die down to 
that corresponding to point 1. The amplitude and frequency correspond- 
ing to point 1 are the amplitude and frequency of the sustained oscillation. 

Point 1 of system B is called a convergent point because disturbances at 
either side tend to converge at these conditions. This is contrasted with 
point 2 of system C, Table 3, which is a divergent point since disturbances 
which are not large enough to give this value of Gd will decay and disturb- 
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Table 3. Typical Loci for Amplitude Sensitive Nonlinear Systems 



Inverted Nyquist 

Diagram Type Nyquist Diagram Diagram 



Stability criteria 



A. Stable system 



-G L 



1 



GM 




kJ 



- TT = CKjcS) 



^S 



Log-Angle Diagram 

201ogio \G(ju>)\ 
= 20 1og 10 |l/Gi,|, 

= -180° - ZG D 




B. System with a 
convergent 
point 



A> 



T 




C. System with a 
convergent 
point 1 and a 
divergent 
point 2; Case 
I, stable for 
small signal 





D. System with a 
convergent 
point 1 and 
a divergent 
point 2 ; Case 
II, unstable for 
small signals 
and very large 
signals 



r*^- 
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ances which are larger will result in oscillations which tend to increase in 
amplitude. 

In system D, point 1 is convergent and 2 is divergent. 

Comparison of the Methods of Presentation. All the methods are 
equally valid. The designer may thus choose the one with which he is 
most familiar and/or the one which best fits the design problem. In the 
inverted Nyquist diagram many of the simpler describing functions are 
bounded, whereas in the Nyquist diagram the describing functions will 
be infinite for some conditions. The other factors that influence the selec- 
tion of one form of the Nyquist diagram are still applicable. (See Chap. 
21.) 

The log-angle method of depicting the stability of the system has advan- 
tages in synthesizing a system containing nonlinearities. It is somewhat 
quicker to plot since G(jui) can be obtained directly from a Bode diagram. 
This method of display also lends itself to use with Nichols charts and 
templates. (See Sect. 7.) 

Frequency Variant Describing Functions. (See Ref. 12.) Describ- 
ing functions which vary with frequency as well as signal amplitude 
will appear graphically like the typical plot of Fig. 9. The describing 
function becomes a surface in three dimensions (magnitude, phase, fre- 
quency), and if this surface is pierced by the frequency locus (also plotted 




Fig. 9. Log magnitude-angle diagram for a nonlinear system with a frequency and 

amplitude sensitive nonlinearity. Intersections (1) and (3) are not significant because 

at these intersections G(jui) and (?z>(w) do not have the same frequency. 
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in three dimensions), the system will be unstable at the frequency and 
amplitude of the intersection. In other words, to have a significant inter- 
section, it is necessary to have the intersection of the (tdO'co, \M |) and 
G(ju) loci occur at the same frequency. This is indicated in Fig. 9 at co 2 . 

Tables of Useful Describing Functions 

Amplitude Sensitive Nonlinearities. Table 4 gives some of the 
more common describing functions for simple amplitude sensitive non- 
linearities, with corresponding graphs in Figs. 10-16. 
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Table 4. Useful Amplitude Sensitive Describing Functions 
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Fig. 10. Describing; function for saturation. 
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Fig. 11. Describing function for deadband. 
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Fig. 12. Describing function for hysteresis. 
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Fig. 14. Describing function for a relay contactor with hysteresis: (a) magnitude, 

(b) phase shift. 
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Fig. 15. Describing function for granularity. Dashed lines are used when granularity 

has a finite number of steps. 
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Fig. 16. Describing functions for variable gain elements, n = m* 



Complex Nonlinearities. Table 5 gives the block diagrams for non- 
linearities which are frequency sensitive or which cannot be conveniently 
separated from frequency sensitive elements. Included in this table are 
the describing functions for several of the nonlinear elements, and graphs 
are given in Figs. 17-19. 

The block diagrams can be mechanized on an analog computer and, 
in general, if extensive investigation is needed, a computer solution is 
recommended. See the instruction manuals of the particular computer 
to be used for details on computer circuits. 



Table 5. Description of Typical Complex Nonlinearities 



Type of Nonlinearity 
1. Motor velocity limiting 

K a = acceleration constant 
K c = viscous damping con- 
stant 
J = motor-load inertia 
T = motor time constant 
= J/KeKa 
M' = M/K e , normalized in- 
put 
Nl = limiting velocity 



2. Motor acceleration limiting 

£| K a , K e , J, T, M' same as above 
u Nl = limiting acceleration 
° TNl = torque limit referred 
to normalized input M' 



3. Backlash (d-c shunt motor 
driving load) 

Jm, Jl = motor and load in- 
ertia respectively 
referred to same 
speed: gearing in- 
ertia divided be- 
tween motor and 
load, depending 
upon location of 
backlash 
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, linear response 



Reflected load torque 



1. Linear machine 
performance ex- 
cept for limiting. 

2. Limiting of motor 
acceleration is 
abrupt. 




l.No gearing 
bounce. 

2. Only viscous 
damping. 

3. Lumped param- 
eter representa- 
tion of gearing, 
motor and load 
(see column 1). 



Gearing, backlash, and load 



General Remarks 
Limiting velocity appears as increased damping; i.e., 
an effective increase in K e and therefore an appar- 
ent decrease in motor gain. In general the decreas- 
ing gain and phase shift of this type of limiting 
tend to make the system more stable but sluggish 
when limiting occurs. The block diagram can be 
extended to higher order systems. Note that in 
modifying the block diagram to suit a different 
situation when a variable is limited, the deriva- 
tives of that variable must go to zero in a physical 
system. The describing function for the simple 
one energy storage motor shown in the block dia- 
gram has been determined experimentally in Ref. 
16 and is given in Fig. 17. The configuration for 
velocity limiting is applicable to hydraulic, d-c and 
a-c servo motors and can be easily mechanized on 
an analog computer. 

Limiting acceleration appears as an increase in mo- 
tor inertia to torque ratio; i.e., a decrease in Kg./ J 
and therefore an apparent increase in motor time 
constant. However, because the damping is not 
affected, the motor low-frequency gain (1/K e ) is 
unchanged. The resulting increase in phase shift 
with this type of limiting can lead to serious per- 
formance deterioration. Torque limiting slows 
the initial response and also the rate of correction 
for overshoots. The latter can result in large over- 
shoots. The describing function of the simple one 
energy storage motor has been determined experi- 
mentally in Ref. 16 and is given in Fig. 18. The 
effect of acceleration limiting can be estimated by 
simply modifying K a to account for the limiting. 
The gain change can be obtained by using the de- 
scribing function for saturation, Table 4. 

The large number of variables in the complete prob- 
lem precludes the derivation of simple describing 
functions. Describing functions have been derived 
for a simplified configuration assuming no viscous 
damping and either (a) a unidirected Coulomb fric- 
tion force (see Ref. 14) or (6) a J m » Jl so that 
the motor motion is not affected by reflected load 
torques (see Ref. 15). Relationships have also 
been determined experimentally (see Ref. 53). 
The difficulty of handling the complex describing 
functions, the restrictions of the basic assumptions, 
and the desirability of using multiple feedbacks in 



_ Describ- 
ing Func- 
tions 
Fig. 17 



Fig. 18 



Kl = effective spring 
constant between 
motor and load 

Dl = effective load 

damping with re- 
spect to fixed ref- 
erence 
B = backlash angle 
KT,R,K e = motor torque, im- 
pedance and veloc- 
ity, constants 



4. Simplified backlash (high 
load damping) 







B vK^~ 


Motor 
(see 3 above) 


«m 









5. Simplified Coulomb friction 



m r Spring constant, K , n 

r— / raramr— £=i 

Friction force < v^mUMMMi 



1 



i + i- 



4F 



7riC|iVi| 
iVj = fundamental of output 



1. J m » Jl, i.e. 
reflected load 
torques have a 
ne gligibl e effect. 

2.\/JlK/D L «1, 
i.e., high load 
damping so that 
the load does not 
coast. 

3. Instantaneous ac- 
celeration of the 
load up to speed 
when the gears 
engage. 

1. Zero load mass, 
Jl = 0. 

2. Zero viscous 
damping. 

3. Friction force 
equal and oppo- 
site to applied 
force up to a max- 
imum of F. 

4. No effect on driv- 
ing motion of re- 
flect load forces. 



practical systems limit the usefulness of this ap- 
proach. The approximations mentioned in the 
text or below in 4 and 5 usually suffice for prelimi- 
nary studies, and a computer study is usually nec- 
essary, for a more thorough investigation. See 
Sect. 7 for a more detailed discussion of backlash 
effects. 

The block diagram can be modified to include (a) 
Coulomb friction by making Dl a nonlinear func- 
tion, (6) speed and acceleration limiting by modi- 
fying the motor blocks as shown in 1 and 2 above, 
(c) multiple backlash by properly dividing the in- 
ertia, springiness, etc. 

This presentation is useful for many types of instru- Table 4 
ment servos where there is little coasting by the Fig. 12 
load. Use the hysteresis describing function, Ta- 
ble 4, for backlash where H = total backlash angle. 



This describing function is useful when Coulomb 
friction is a major influence on performance (see 
Ref. 17). Magnitude and phase are the same as 
for a simple time constant where frequency 
Ta> = IF/(ttK\Ni\) A more exact analysis can be 
made by using analog computers and introducing 
friction as a nonlinear damping term, Dl, i.e.; 




Fig. 19 
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Fig. 17. Describing function of a single servomotor subject to velocity limiting; 
M' = normalized magnitude of input, radians per second; Nz, = saturated speed, 
radians per second (maximum speed); T = unsaturated motor time constant, seconds 

(Ref. 16). 
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Fig. 18. Describing function of motor subject to acceleration limiting (Ref. 16). 



a.u 1 1 1 1 1 1 1 






^ - *i^ 1 '* 


OR •*■*''* — «•" ■ 


^? — — — ■"* 




^ ^^^^ Phase shift 




nfi v^ -,>**"* 




r *^ 


2 /^ 


2 ^ 


04 > /^ 




7>' 


i > 


2^ 


2 ^ 




> X 


L 


7 


f 



-10 

-20 | m 
-30 2J 
-40-|"°. 
-50 J C? 
-60 = .- 

<n TO 

-80S 
-90 



0.5 1 1.5 2 2.5 3 3.5 4.0 

Fundamental dimensionless output motion, -£ | jyj 

Fig. 19. Describing function for Coulomb friction. See Table 5 for definition of terms. 



25-34 



FEEDBACK CONTROL 



Simplifying Complex Nonlinearities 

Separation of Amplitude and Frequency Sensitive Elements. It 

is obvious that the nonlinearities of Table 5 can be simplified by a number 
of approximations to the point where the simpler amplitude sensitive 
describing functions can be used. Note that many of the complex non- 
linearities consist of a combination of a simple nonlinearity and frequency 
sensitive element(s). Typical of such a case is the system with backlash 
shown diagrammatically in Fig. 20a. 






G(s) 



E 3 






hrOH 



JutS 



K c 






G n — 



Kr 



$1 



1 
Jr.S 



L D J 



(a) 



i£ 



GGi 



'' + 



o 



G, KlR /-, 



(b) 

Fig. 20. Block diagram for a simple system with backlash Gd and G(s) equal to con- 
troller elements: (a) the conventional diagram arrangement; (b) the diagram rearranged 
so that the amplitude and frequency sensitive portions of the system are separated. , 



If it is recognized that the input to the nonlinear portion of the system 
will be essentially sinusoidal, then the describing function for the dead- 
band (Table 4, Fig. 11) can be used and substituted directly into the block 
diagram as a gain Gd- To analyze the system conveniently it is necessary 
to rearrange the block diagram so that the nonlinearity is separated from 
the frequency sensitive elements and can be treated by the usual techniques 
of graphically presenting amplitude sensitive describing functions. 
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Let 



Gm = 



G L = 



( JtnR \ 

K e 8[ S+ 1) 

\K T Ke ) 

Kn/D l 



£■♦■) 



Then rearrange the block diagram to appear as in Fig. 206. In this illustra- 
tion the amplitude and frequency variant portions of the systems have 
been separated and the usual type of describing function analysis can be 
used to determine if the system will be stable and if not what the amplitude 
and frequency of oscillation will be. 

Method of Equivalent Coefficients. In complex systems, it becomes 
difficult and sometimes useless to attempt to separate the amplitude 
sensitive and frequency sensitive elements in the system so that the method 
of analysis described earlier in this section under Theory of Describing 
Functions can be used. For instance in the above example the variables 
in which one is really interested, C and R, are buried in the block diagram. 
It is thus difficult to determine: (1) whether the overall system will have 
satisfactory performance, and (2) how to modify the compensation to 
improve performance. 

The technique of using an equivalent coefficient is: (a) to recognize that 
many nonlinearities appear as a gain change in the system, and (b) to 
combine this gain change with existing gains in the system to form an 
equivalent gain or coefficient. Once knowing the range of gain to be ex- 
perienced, the system can be designed to be as insensitive to such a varia- 
tion as is desirable. 

A way to avoid the difficulty in the previous example lies in this approach. 
It was pointed out that the describing function of the dead space can be 
considered directly in the analysis. This describing function is in series 
with the spring constant and can be combined to yield an equivalent spring. 
Thus, the system will seem to have a very soft spring at low angular dis- 
placements and an increasing spring constant (approaching the actual 
spring constant) as the angular displacement increases. This equivalent 
coefficient can then be considered as a constant in the remaining analysis. 
In this case the major effect is the reduction in the load resonant frequency 
and it becomes necessary to make the system less sensitive to load resonant 
frequency to avoid difficulties. 



25-36 FEEDBACK CONTROL 

It is necessary to consider a number of different spring constants to 
make sure that no unstable points exist, but this is usually not too difficult 
a task although it can be somewhat time consuming. (See Ref. 18.) 

Example. As shown in item 3 of Table 5, the equation for backlash 
from armature voltage to output shaft rate is 

S0 L 

Va = K' l Kt/R 

K e K Tl o , f/r , , ^, ■ D LKeKTl , „, \ ^ , K e K T l 



JmJ L s 3 + \JmD l +Jl- j ~Y+ [(Jm+Jl)K' l +-^-^]s+K' l [d l + 



R J 

where K'l = the equivalent spring constant = KlGd- The value of Go 
is obtained from Fig. 11 with the argument B/2(6m — 6l) rather than 
B/2M. 

The complete system transfer function including the above equations 
can then be analyzed for several values of K'l- The actual value of Gd 
has to be considered only if the magnitude of the input to the backlash is 
wanted. 

5. METHODS OF ANALYSIS: PHASE PLANE, GRAPHICAL SOLUTION OF 
SYSTEM EQUATIONS 

General 

This is essentially a heuristic presentation of the phase plane method. 
As a consequence attention will be directed at the areas of application and 
only the most rudimentary explanation of the techniques of constructing 
the diagrams will be provided. At its present state of development this 
technique has only limited utility in system synthesis; however, phase 
plane techniques have received some use in the conception and display of 
schemes for nonlinear compensation, (see Refs. 22 and 23.) 

The reader is referred to Refs. 20, 21, and 27 for details beyond those 
provided here. 

Definitions. The phase plane has the coordinates of velocity (usually 
the ordinate) and position (usually the abscissa) of the system. The solu- 
tions of the differential equations are plotted on this coordinate system. 

The locus of a solution to the differential equation is called a phase 
trajectory or simply trajectory. A series of solutions or trajectories is referred 
to as a phase portrait. 

Limitations. Analysis by the phase plane method is limited to : 

1. Second order (single degree of freedom) systems. 

2. Autonomous systems (time does not appear as a parameter in any 
of the coefficients of the system). 

3. Systems with impulse, step, or ramp inputs or driving functions. 
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The limitation on the order of the system is severe, but it is possible to 
approximate a limited number of practical systems by a second order 
equation for purposes of preliminary analysis. Methods have been pro- 
posed for extending the technique to higher order systems but have not 
received wide use. (See Refs. 24 and 25.) 

Basic Equations. The basic equations that can be treated by phase 
analysis are of the form : 

d 2 x dx dx 

(6) — + fi(x, x) — + hi*, x)x = 0, x = — • 

dr dt dt 

By substituting dx/dt = y, eq. (6) can be reduced to a set of first order 
equations: 

dy 

— = N(x, y), 
dt 

dx 

— = D(x, y), 
dt 

and eliminating time by division yields 

dx dy 

(8) 



N(x, y) D(x, y) 

Significant Characteristics of Phase Portrait 

Table 6 describes a few of the significant characteristics of phase tra- 
jectories. Identifying these characteristics will be useful to the engineer 
in interpretation of the phase trajectories. 

Areas of Use of Phase Plane Method 

Analysis. The graphical techniques of plotting the phase trajectories 
make the phase plane method particularly useful for systems with second 
order nonlinear equations of motion. Although the availability of analog 
computers has greatly reduced the need for such hand methods, there still 
remains a need for generalizing analysis. The phase plane analysis often 
can provide this generalization. 

Presentation of Data. The phase plane has found some use in presen- 
tation of analog or actual equipment results. In such cases, the system 
does not need to be limited to second order. Of course the interpretation 
becomes more difficult the higher the order of the system. Such plotting 
techniques have been made even more meaningful when the display is on 
a cathode ray oscillograph by intensity modulation. Timing pulses can 
be indicated by brightening or dimming the trajectory. 



Table 6. Significant Characteristics of Phase Portrait 



Type Description 

1. Nodal The trajectories converge or radiate from the node in such a 
point manner that the direction of the trajectory approaches defi- 
nite limits as the nodal point is approached. The node is 
stable if the paths converge on the node and unstable if the 
paths radiate from the node. This is a singular point, i.e., 
a point where eqs. (7) are equal to zero. 

2. Focal The trajectories converge or radiate from the focus on spiral 
point paths. As for the node, if the paths converge, the focus is 

referred to as stable; if the paths spiral outward, the focus 
is referred to as unstable. This is a singular point. 

3. A cen- Closed trajectories about a point. This is a singular point, 
ter 



4. Saddle Trajectories converge toward the saddle point and then diverge 
point except for the special case when the initial conditions are 

such as to fall on the trajectory that goes into the point (the 
converging separative) . This is a singular point. 



5. Limit A limit cycle describes the oscillation in a nonlinear system, 
cycles A stable limit cycle is a closed path to which adjacent trajec- 
tories converge. When the trajectories diverge from a closed 
path in the phase plane, the path is called an unstable limit 
cycle. 

6. Hard It is necessary to excite the system beyond a finite bound in 
oscil- order to obtain self-sustained oscillations. The boundary 
lations will be an unstable limit cycle. 



Typical Trajectories 



^V 



-/&- 



Corresponding Conditions in 
Linear Second Order System 
Stable node, negative real 

roots 
Unstable node, positive real 

roots 



Stable focus, complex roots 
with negative real parts 

Unstable focus, complex roots 
with positive real parts 



to 

Oi 

CO 
00 




Zero damping 



Both roots real, one negative 
and one positive 



A stable limit cycle about an unstable focal point 




Separatice 



Limit cycle None 
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7. Soft Self -sustained oscillations can be started with an infinitely small 

oscil- excitation. Soft oscillations start from unstable nodes or fo- 

lations cal points. 



See figure for limit 
cycle 



None 



None 
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Once the analyst has set up the equations for the phase plane, an analog 
computer can be used to plot the trajectories. In this manner, one can 
maintain the generality of the phase plane analysis and avoid the ennui 
of extensive hand calculation. 

System Synthesis. Because the graphical presentation often makes 
interpretation of results easier, the phase plane method has been looked on 
with favor by many. A number of authors describing work on "optimum 
controls" have made extensive use of the phase plane in presenting their 
results. (See Sect. 7 for details.) However, the limitations on the order 
of the system equations hamper work on any but the simplest systems. 

Ku and a number of others have extended the phase plane to phase 
space. (See Refs. 24 and 25.) This is essentially a multidimensional 
plot allowing solution of higher order equation. Phase space methods have 
received only very limited use to date. 

Analytical Methods of Constructing Phase Plane 

Direct Method of Solution. If the equation of motion of the system, 
eq. (6), can be integrated to obtain time solutions for x, then a; as a function 
of x can be obtained by eliminating time from the individual solutions, 
and the relationship between x and x may be plotted directly. 

Indirect Method of Solution. If the equations of motion cannot be 
integrated to obtain time solutions for x and x, a new differential equation 
in terms of dx and dx may be formed and solved to give a; as a function of 
x directly. The equations in this case reduce to the form of eq. (8). 

Example. Simple Relay Servo. Equations (13), Sect. 6, describing the 
operation of a relay servo for a step input are repeated here for convenience. 

d 2 x dx 



d 



T 

d 2 x dx 

— + — = 1, x < 0. 

dr dr 

Substitution of y = dx/dr, then dividing by y = dx/dr, and recognizing 
that (dx/dr)/(dy/dr) = dx/dy yield : 

dy 1 

— =--—!, x>0; 
dx y 

(9) 

dy 1 

— = + - - 1, x<0. 
dx y 
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The variables can be separated in these equations and integrated : 

y 



(10) 



-It 



+ 2/ 



dy, x > 0; 



= I dy, 

J 1 - y 



x < 0. 



These integrals can be found in any good table of integrals, and the 
function can be plotted on the phase plane for different constants of in- 
tegration. When plotted, the trajectories of eq. (10) would appear similar 
to the sketch in Fig. 21. 




Fig. 21. Phase trajectories for a simple relay servo. Each trajectory is for particular 
constant of integration of eq. (10). 

Obtaining Time from a Phase Plane Plot. It is possible to obtain 
time, t, from a phase plane plot even though the original characteristic 
equation of motion cannot be solved for a; and a; as functions of time. To 
do this use the relationship : 



(id 



J J dx/dr J x 



dx. 



dx/dr 

Equation (11) shows that if the phase portrait is replotted with 1/x as 
the ordinate and x the abscissa, the area under the resultant curve re- 
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presents time. This method makes it possible to obtain plots of x and x 
versus time. Graphical methods are also available for obtaining time from 
the phase portrait. (See Ref . 26.) 

Graphical Methods of Constructing Phase Plane 

When the original characteristic equation of motion is nonlinear, the 
integration of the equation obtained by the above method is difficult or 
impossible. Graphical methods exist for solving the equation for a direct 
plot of x versus x. One of the most useful methods is the method of iso- 
clines, described in Refs. 20 and 23. Other graphical methods are also 
available, e.g., Lienard's, arc-segment procedures. 

Method of Isoclines. Equation (8) can be written in the form 

dx Nix, y) 
(12) — = • 

dy D(x, y) 

Equation (12) is the slope of the phase trajectory. By setting eq. (12) 
equal to a constant, the equation for the locus of a constant slope can be 
obtained. One can then strike off lines of the proper slope along the locus. 
After constructing sufficient loci of constant slope, the phase trajectories 
can be sketched. 

Example. Simple Relay Servo. Equations (9) define the loci of constant 
slope for the relay servo given in the previous example. 

dy 1 

(9a) — = 1, x> 0; 

dx y 

dy 1 
(9b) — = - - 1, x < 0. 

dx y 

Equation (9a) set equal to a constant provides the loci in the right half- 
plane. Equation (96) describes the left half-plane. 
For a +45° slope eq. (9a) becomes: 

1 - 

i = i, y=-h 

y 

Several values are tabulated in Table 7. The isoclines are constructed 
in Fig. 22. 
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Table 7. Values of Isoclines for Simple Relay System 

Value of y 



Slope 


Left Half-Plane 


Right Half-Plane 


0° 


1 


-1 


+30° 


0.634 


-0.634 


-30° 


2.36 


-2.36 


+45° 


0.5 


-0.5 


-45° 


±W 


=fcw 


+60° 


0.366 


-0.366 


-60° 


-1.37 


1.37 


+75° 


0.211 


-0.366 


-75° 


-0.366 


0.211 


+90° 







-90° 









Fig. 22. Construction of a phase traj ectory by means of isoclines for a simple relay servo. 
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6. OTHER METHODS OF ANALYSIS 

Differential Equations: Analytical Solutions 

An often useful method of analytically obtaining the transient perform- 
ance of a simple but useful class of nonlinear systems is by piecewise 
linearization. Although the type of nonlinearity that can be treated is 
restricted, higher order systems can be handled. (See Refs. 31, 32, and 
33.) Other analytical methods of obtaining transient solutions are de- 
scribed in Sect. 5 and in Refs. 27, 29, and 30 and their bibliographies. 

Piecewise Linear Systems. Many nonlinear control systems are 
linear in well-defined areas of operation. At the boundaries of these linear 
areas are discontinuities which make the system, when considered as a 
whole, nonlinear. For such systems the linear differential equations can 
be solved between boundaries and the boundary conditions matched to 
obtain a complete solution. 

Since it is generally desirable to obtain a solution under steady-state 
conditions (or at least as steady-state conditions are approached), the 
process using differential equations becomes quite laborious if there are a 
number of reversal points. This is true even for a simple second order 
system, and the process becomes more unwieldy for higher order systems 
where more than two initial conditions are required at each reversal point. 

Normalized Performance Charts. Kahn avoided some of the labor 
in the differential equation approach by using a semigraphical approach that 
recognizes the fact that the initial conditions at each boundary point are 
a function of the velocity at the previous boundary and the time between 
boundaries. (See Ref. 32.) This method loses its value for systems of an 
order higher than second. Under these conditions more than two di- 
mensions are needed to represent the curves. For a higher order system, 
it is necessary to have more than one initial condition for each boundary; 
for example, on a third order system, it would be necessary to have initial 
conditions representing both velocity and acceleration. The higher de- 
rivatives would fall in the third dimension. 

Summary of Steps in Piecewise Linear Analysis. 

1. Prepare the complete system equations. 

2. Break the complete equations into a set of linear equations represent- 
ing the system operation between discontinuities. 

3. Determine the boundary conditions at the discontinuities. 

4. Nondimensionalize the equations as much as practical. 

5. Rearrange the equation or change the dependent variable to make the 
dependent variable independent of the input function at the discontinuities. 
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6. Obtain the solutions to the equations. These may often be best 

presented graphically. 

Relay 



J — t 9- 
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Motor 






y e 
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T m s + 1 










— e 






> 













Fig. 23. Block diagram of simple relay servo. 

Example. Piecewise Linear Relay Servo. A typical block diagram for a 
relay servo is shown in Fig. 23. For the motor of the system of the figure, 

d 2 c dc 

m = K v T m — + K v — • 
dt 2 dt 

From the characteristics for a simple relay 

m = — V f or e < 0, 

m = V for e > 0. 
Therefore, 

d 2 c dc 

K v T m — + K v -= -V, for e < 0; 
dt 2 dt 

d 2 c dc 

K v T m — + K V -=V, for e > 0; 
dtr dt 

e = r — ■ c. 

A typical transient to a step input for the servo of Fig. 23 appears in 
Fig. 24. The driving voltage in the motor is reversed at each of the zero 




Time 



Fig. 24. Typical response of simple relay servo to a step command. 
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error points 1, 2, 3, 4, 5, etc. Substitution of c = r — e and normalizing 
the variables by substituting 









T = 


' \Tj 




yield for a step 


input : 


X = 


\T m V/ 




(13) 




d 2 x 
d^ 
d 2 x 
aV 


dx 
dx 


-1, for 

1, for x 


x > 

<0, 



The solutions to these equations are obtained by taking the Laplace 
transform of eqs. (13) and determining the inverse transform. The trans- 
forms of eqs. (13) after the switching at r n are 

(14a) s 2 X(s) + sX(s) = [-l/s + sx(r n ) + x{r n ) + x{r n )}e~ TnS , x < 0; 

(146) s 2 X(s) + sX(s) = [l/s + sx(r n ) 4- x(j n ) + x(r n )]e~ TnS , a; > 0; 

where x(T n ) = value of x at r n , 
X{8) = £^(r)], • 

x(r n ) = derivative of x with respect to r at t„, 
r n = normalized time of the nth switching point. 

Notice that with the exception of the first closure of the relay (in the 
region — 1 of Fig. 24), x(r n ) = at the switching point and only x{r n ) 
affects the characteristics of the transient. The velocity just before the 
relay switches must equal the velocity just after the relay has switched. 
This is the boundary condition relating the two eqs. (13). To obtain a 
solution to eqs. (13), it is then necessary to apply the initial conditions at 
each reversal point. 

For eq. (14a) the inverse transform yields, when x(r„) = 0, 

(15) x = - exp [-t + r„] — (t — t„) + 1 + x(j n )[l - exp (-t + t„)], 

T n+1 > T > T n , 

where r n+ i = normalized time at the n + 1 switching point. 

Equation (15) is dependent only on the time from the last reversal and 
the initial velocity. The time to reach the next reversal can be calculated 
by setting eq. (15) equal to zero and solving for the time difference r n+ i — 
r n . This equation is 

, , . s exp ( — T n+1 4- T„) + T n+ 1 — T n — 1 

(16) x(T n ) = ■ 

1 - exp (-r n+ i + r„) 
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The velocity at the next reversal can be obtained from the derivative of 
eq. (15) at r n+1 : 

(17) x(r n+1 ) = exp ( — t„+i + T n ) — 1 + x(r n ) exp (-r n+1 -f- r n ). 

Equation (146) can be solved similarly. Since the system is symmetrical, 
the equations corresponding to eqs. (16) and (17) will be respectively 
identical except for sign. 

Kahn avoided the labor of obtaining repetitive solutions to eq. (15) 
by plotting eqs. (16) and (17) as shown in Fig. 25. The transient x(t) 
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Fig. 25. Plots of piecewise linear relay servo (a) performance charts and (6) method 
of sketching time response from performance chart data. 

can be quickly sketched by laying off the slopes, x(r n ), at the switching 
times, r n . The method can be extended to more complex nonlinear func- 
tions than described, but practically the system must be limited to second 
order. 
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Methods for Analysis for Variable Linear Systems 

A differential equation with time-varying coefficients can be linear; how- 
ever, the useful frequency response and root locus methods are primarily 
applicable to constant coefficient linear differential equations and cannot be 
extended simply to the case with time-varying coefficients. This section 
presents several special techniques which can be used with variable linear 
systems. 

Equivalent Coefficients for Variable Linear Systems. This method 
is useful where the variations are slow with respect to system response. 

The designer can reduce the system to a series of constant coefficient 
cases as follows: Consider the general case of eq. (18). 

d n x(t) 

(18) (a n0 + a nl t -\ ) in 

dt n 

dx 

-\ h (a 10 + a n t -{ ) — + (aoo + aoit H )x 

dt 

d m y(t) dy 

= (b m o + b ml t + • • •) — — + • • • + (&io + b n t + • ■ •) -r 

dt m dt 

+ (&oo + b 01 t H )y, 

where the a^, b{ k are the constants for t to the /cth power in the coefficient 
of the ith derivative. 

To investigate the performance at an instant of time, t 1} substitute ti 
for t in the coefficients. The resulting constant coefficient linear differential 
equation can be treated by the usual linear constant coefficient techniques. 

d n x(t) dx(t) 

(19) An — .+...+ Al — +AoX(t) 

d m y(t) dy 

= B m + • • •+ B x — + B y(t), 

dt m dt 

where A n = a n0 + a nl t x + a n2 ti 2 +•'••, 
B m = b m o -f- b m iti + & OT 2^i "I - '' - - 

Ultimate Stability of Variable Linear System. Kirby pointed out 
a method of exactly determining the ultimate stability of eq. (18) as 
t — > co. (See Refs. 34 and 35.) Basically one forms a new equation from 
the coefficients of the highest order of t in the coefficient of each derivative, 
i.e., 

(20) a nk s n H 1- a lk s -f a Qk = 0. 
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The stability of eq. (20) can then be determined by the Routh or another 
stability criterion. To be applicable the order of t in the coefficient of the 
highest derivative must be of as high an order as any t in the coefficients 
of the remaining derivatives. If the same order of t does not exist in the 
coefficient of a derivative, then the value of a is considered to be zero for 
the corresponding term in the hew equation. It should be noted that the 
Routh (or similar stability criterion) test for stability then cannot be ap- 
plied directly to eq. (20) . 

Because it indicates only the ultimate stability as t — » «>, the method 
is limited. 

Adjoint Computing Method. The adjoint computing method is val- 
uable in cases where (a) the system is linear time- varying and (6) the 
performance is wanted at only one instant of time as affected by disturb- 
ances at other times. 

This is a powerful analytical tool and is particularly well suited for use 
on analog computers. 

The method is described in Chap. 24. 

7. NONLINEAR SYSTEM COMPENSATION 

This section covers (a) compensation of a nonlinear system by linear 
and nonlinear elements and (b) introduction of intentional n on linearities to 
improve the response of an originally linear system. 

Because of the difficulty of generalizing a nonlinear system, the majority 
of the published work has been either on second order systems or in terms 
of a specific application. For this reason, in the following, emphasis will 
be placed upon the ideas involved rather than on the details of the method 
of analysis. 

Compensation of Slowly Varying Nonlinearities 

Missiles, engines, and many industrial processes have wide variations 
of system parameters. These variations are often slow compared with the 
other variations of the controlled quantities about a particular operating 
point. 

Such changes in system characteristics are usually the result of the 
variation of an independent variable. This is the case in a ground-launched 
missile. The aerodynamic coefficients vary with altitude and speed, and 
since these coefficients determine the transfer function, the missile dynamic 
characteristics will vary as the missile climbs and accelerates. Although 
the change in gain from control surface motion to missile attitude can be 
several thousand to one, these changes may be relatively slow compared 
with the response of the system controlling the missile attitude. 

Conversely, a slow variation in characteristics can be the result of the 
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change in the dependent variable. For instance in a turbojet engine speed 
regulator, the fuel to the engine is varied to maintain speed. The transfer 
function of the engine varies widely with engine speed — as much as 10 to 1 
from idle to top speed. However, it normally requires several seconds to 
accelerate an engine of this type through this speed range which is slow 
compared with the speed regulator response. 

Technique for Compensation. For systems which are of the slowly 
varying nonlinear type, the necessary control characteristics can be de- 
termined by the following process: 

1. Reduce the system equations to linear, constant coefficient form by 
the methods of Sects. 2 and/or 6 at a particular point in the range of opera- 
tion. 

2. Synthesize a satisfactory control transfer function at the particular 
point by the methods of Chap. 23. 

3. Repeat the procedure at a sufficient number of points through the 
operating range to define thoroughly the variation of the control transfer 
function that is necessary to maintain ideal performance over the range of 
operation. 

4. Determine the simplest method of manipulating the control transfer 
function and still satisfactorily approximate the values determined in 3. 

The change required in the control may be a simple gain change or it 
may require the modification of time constants. Considerable simplifica- 
tion may result from an ingenious choice of the method for obtaining the 
control changes. For instance, the variation of the missile characteristics 
was with altitude and speed. The control characteristics could therefore 
also be changed with altitude and speed; however, it may prove simpler 
to program the control changes as a function of time. If the missile path 
is known in advance, the change in altitude can be rather closely predicted 
as a function of time. For the engine control, it. is obvious after some 
parameter study that the gain variation of the engine is approximated by 
an exponential function of fuel flow or a function of compressor pressure. 
Solutions commonly adopted, therefore, use either a nonlinear fuel valve 
in the controller or manipulate controller gain with compressor pressure 
to compensate for the variations in engine gain. The particular method 
selected depends upon many factors involved in the mechanization of a 
particular control. (See Ref. 36.) 

Practical Considerations. If the rate of change of the coefficients 
approaches the response of the system, the actual system must be analyzed 
in more detail. The system can become unstable if the proper combina- 
tion occurs. Of equal importance, if the variation is rapid enough, the 
system can actually operate satisfactorily through a zone which is un- 
stable if analyzed on a small change fixed system basis. In the latter case, 
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the analysis based on the assumption of a slowly varying nonlinearity 
would probably result in a stable system. However, the result of such an 
analysis would probably be an overdesigned system. 

Compensation of Rapidly Varying Nonlinearities 

A nonlinearity which changes system characteristics at a rate which is 
appreciable with respect to the response time of the system is defined as a 
rapidly varying nonlinearity. 

All physical systems have limitations and/or restraints. These may 
take the form of limitations on power, torque, amplifier output, speed, etc., 
or they may be simply the result of limited precision and appear as back- 
lash, deadband, hysteresis, etc. Relay servos and other systems of this 
class represent a special case wherein the nonlinearity is introduced in- 
tentionally. The purpose of such a system may be, as is the case of the 
relay servo, to simplify the system, or the nonlinearity may be necessitated 
because of other system considerations. Regardless of the reason for their 
presence in the system, if these nonlinearities have a marked change in 
characteristics with variations of an independent variable in the range of 
interest, they are rapidly varying and must be considered as an integral 
part of the analysis of the system. 

Rapidly varying nonlinearities can also be a function of the independent 
variable; however, the following applies primarily to nonlinearities which 
are junctions of the dependent variable. 

Compensation with Linear Elements : Use of Describing Function 
Techniques. Methods of predicting the necessary system compensation 
by frequency response techniques which are applicable to linear systems 
have been extended to nonlinear systems with limited success. There 
are certain significant differences between the frequency response of a 
linear and a nonlinear system. For a linear system, the closed loop response 
is defined by a single set of values and these can be uniquely related to 
the transient response. Design criterion in terms of the characteristics of 
the frequency response can therefore be directly interpreted into terms of 
the transient response. For a nonlinear system in which the nonlinear 
element has been replaced by the describing function, a different closed 
loop frequency response is obtained for each value of Go, and therefore a 
single frequency response does not represent the system during a transient 
which passes through the nonlinear region. Even assuming that the de- 
scribing function adequately respresents the nonlinearity, it is difficult 
under such conditions to specify a satisfactory design criterion in the 
frequency domain which is significant in terms of the transient response. 

No generally satisfactory frequency response design criterion for nonlinear 
systems presently exists. 
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However, since the values of the gain and phase margin and M m and o) m 
have received considerable use in linear analysis, a knowledge of the variation 
of these quantities is a valuable link between the frequency response methods 
and the describing function methods. Although of more qualitative than 
quantitative use, it is often valuable to plot these quantities versus the 
magnitude of the input variable. 

Example. If the M m appears too severe, the frequency locus can be 
adjusted to give the desired characteristic. Shown in Fig. 26 are the w m 




0.2 0.3 0.4 0.5 0.6 
Normalized input to relay B/2M 

Fig. 26. The maximum modulus M m and the frequency of the maximum modulus « m 
of a simple relay servo as a function of the argument of the describing function for a 
relay. The operating conditions and the servo frequency response are given in Fig. 27. 
The uncompensated characteristics result from setting the system gain sufficiently high 
to meet sensitivity requirements, i.e., setting B. Maintaining the system gain and 
adding a 3 to 1 lead gives the compensated characteristics. (Note that in practice there 
would have to be a 3 to 1 increase in amplification to compensate for the 3 to 1 attenua- 
tion that a passive lead network would have.) 

and M m of a relay servo before and after compensation. Figure 27 shows 
how G(jo>) locus has been changed to improve system response by meeting 
a criterion of a maximum M m ~ 2. The boundary of such an improve- 
ment can be quickly obtained by overlaying a Nichols chart with a graph 
of the Gd locus and observing the path of the G(jui) locus on the Nichols 
chart as the origin is moved along the Gd locus. (See Fig. 28.) If a maxi- 
mum M m criterion is being used, the boundary of this M m for all values 
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Servo motor response 




Fig. 27. Log magnitude-angle diagram of a simple relay servo; operating conditions 
V/B — 6 db; H/B = 0.5, Curve (a) is the uncompensated system, Curve (6) is the 
system compensated with a 3 to 1 lead. The cross-hatched region is the necessary 
modification to the uncompensated system to meet an M m criterion of M m < 2. For 
the purposes of illustration a normalized response has been used for the servo motor 
and only the ratio V/B has been defined. 



of Gd can be quickly sketched on the G(ju>) graph. If sufficient work of 
this type is performed, templates for several values of M m can be built. 
The necessary compensation networks can be determined either by trial 
and error or by more elaborate methods discussed in Chap. 23. 

Compensation for Relay Servomechanisms. Describing function, 
phase plane, and piecewise linear analyses have all been used extensively 
to determine the necessary compensation. (See Refs. 11, 18, 24, 31, 32, 
and 44.) The describing function method is the most useful for higher 
order systems. The techniques used in the preceding example can be 
easily extended to higher order systems. The describing function analysis 
and experiment normally check within engineering accuracy. (See Refs. 11 
and 41.) A maximum M m < 2 criterion is generally typical in the design 
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Uncompensated locus 




y*> 

L. Nichols chart M 

I contour for M= 2 (6 db) 



Fig. 28. Log magnitude-angle diagram showing a method of estimating the necessary 
compensation by overlaying with a Nichols chart. The origin of the Nichols chart is 
first placed at B/2M = 0.67 and the M contour is sketched on the log magnitude-angle 
diagram. This is noted as curve (a), where M m = 2 has been used for illustrative 
purposes. The complete area of necessary compensation can be found by moving the 
origin of the Nichols chart along the describing function locus. Another location is 
shown at B/2M = 0.6, and the M contour is curve (b). The coordinates of the Nichols 
chart are shown as dotted center lines. 



of relay-positioning systems. However, experience with the condition 
of the particular application may dictate a different value for maximum 
M m or a different criterion. Because of the phase lag of a relay with 
hysteresis, lead compensation is quite useful. Two forms of such compensa- 
tion are tandem lead networks and rate feedback. The latter can be ob- 
tained either from a tachometer or from motor back electromotive force. 
Nonlinear compensation can also be used to achieve better performance for 
a particular type of input. See the paragraphs on Optimum Switching 
Functions later in this section for details. Such compensating networks 
must be used with care if more than one type of input is to be encountered, 
because the performance will vary with the form and magnitude of the 
input function. (See Refs. 42, 43, and 45.) 

Compensation for Saturation. For a large number of systems, it is 
necessary to follow a relatively smooth input within a very small error. 
In order to provide economical components, the linear operating range 
of these components is usually very little beyond that necessary to follow 



Table 8. Typical Types of Saturation Effects and Methods of Compensation 



Type 
of Non- 
linearity 
Case I, 
pre- 
ampli- 
fier 
satu- 
ration 



Case II, 
power 
ampli- 
fier 
satu- 
ration 



System Configuration and Characteristics 
Block diagram 



-^H=/H_^J-r 



Open loop characteristic 




■ytl 



— / — Gi 



Gi = integral compensation 

G2 = power element 

Same open loop characteristics as Case I 



Effect on System Performance 
For an unconditionally stable system with sat- 
uration, the relative response will be slower for 
large step inputs. As the step input level is 
increased for a conditionally stable system, 
the system will begin to exhibit less stable 
characteristics until a critical value is reached 
above which self-sustained oscillations will oc- 
cur. For moderate saturation, the overshoot 
may actually be less than for the linear system 
although the settling time will be increased. 
Saturation acts like a gain reduction in the 
system and for saturation to cause system in- 
stability, instability must be predicted on a 
linear basis for reduced gain. Oscillation fre- 
quency and amplitude can be estimated within 
20 to 30% by describing functions. The pres- 
ence of noise with the input signal causes an 
effective increase of saturation beyond that 
predicted for the input signal by itself. This 
effect causes an increase in the closed loop 
phase shift with respect to the input signal. 
See Ref. 7 and Table 1. 

Same effect as Case I but a reduction in the over- 
shoot with moderate saturation has not been 
observed. The degree of saturation (ratio of 
saturation level to input to element if the sys- 
tem were linear) necessary to start self-sus- 
tained oscillations in a system will vary with 
the location of the nonlinearity in the system. 
A difference as great as 10 to 1 between the sat- 
uration from a step of r needed in the pream- 
plifier and power amplifier has been noted. 
(See Ref. 18.) In all the cases considered in Ref. 
18, it was necessary to have sufficient satura- 
tion so that the gain was reduced to the point 
where there was negative phase margin at gain 
crossover; however, with preamplifier satura- 
tion it was necessary to exceed this level of 
saturation considerably to cause self-sustained 
oscillations. 



Possible Methods of Compensation 
(a) Eliminate or reduce the integral compensation for large signal 
inputs; e.g., typical magnitude and phase angle curves are 
shown at the left. It is obvious that if the gain is reduced suf- 
ficiently, the region of negative phase margin will cause insta- 
bility. However if a nonlinear compensating network is used 
which for large errors eliminates or reduces the integral com- 
pensation, satisfactory performance can be obtained. The 
dotted line shows the frequency response after such a change. 
There are a number of methods by which such changes in the 
compensation can be obtained. (See Refs. 18, 38, and Table 
10.) The circuit constants are normally set experimentally. 
(6) Modify the basic system operation for large signal levels. This 
is essentially an extension of (a). However the basic mode of 
operation is also changed. Examples are dual-mode servos 
wherein the mode of controlling the power element is changed 
with signal level, and two-speed servos wherein the feedback 
signal gain is lowered. (Actually, in practice, the takeoff is 
from a different speed shaft which gives rise to the appellation 
two speed.) (See Refs. 39 and 40.) Often the signal used to 
switch the feedback signal is used to modify the compensation 
networks. 



(a) See Case I (a) and (6). Power amplifier saturation is similar 
to torque saturation for which the dual servo techniques have 
been developed. (See Refs. 23, 40, and Table 10.) 

(6) Use of tachometer feedback around the saturation is effective. 
(See Case III.) 

(c) If in place of Gi the integral compensation can be accomplished 
by a filter (lead) network around the saturating element, the 
system can be so designed that, as the system saturates, the 
compensation automatically becomes less and the system will 
not become unstable with saturation. (See Ref. 2.) 




Case IV, 
satu- 
ration 
in feed- 
back 



(?i = integral compensation 

G2 = power element 

G3 = tachometer feedback 




For a conditionally stable system saturation, the 
gain is reduced in the tachometer loop lower- 
ing the crossover frequency, at, which lowers 
the phase margin at the position loop. There 
are two possibilities: (1) The phase margin at 
the normal position loop crossover frequency, 
oic, will be lowered until the system becomes 
unstable at the normal crossover frequency, 
oi c . (2) The tachometer loop becomes ineffec- 
tive before (1) occurs, and the position loop 
gain will be lowered forcing the crossover 
frequency down into the region where the 
phase margin goes negative on account of the 
integral compensation. • The effect depends 
upon the constants of the particular system be- 
ing considered. For (1) the oscillation fre- 
quency will be approximately the crossover 
frequency. For (2) the frequency of oscilla- 
tion will be closer to the integral compensation 
time constants. 



Effect similar to Case I. 



(a) Instability is not normally a serious consideration. 

(b) The problem can generally be avoided by achieving the com- 
pensation by a filter in the tachometer feedback rather than 
in tandem elements in G\. 



(a) Eliminate saturation if possible. 

(6) See Cases 1(a) and (6) and 11(c) above. 



Same open loop characteristics as Case III 
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the input within the maximum allowable error. When such systems are 
synchronized on a new operating condition or subjected to violent disturb- 
ances, they will inherently be highly saturated. This leads to reduced 
performance. In addition, because such systems often use integral com- 
pensation to achieve high values of low-frequency gain, this can also lead 
to serious overshoots and the attendant longer settling times or even in- 
stability. Normally the requirement on allowable error is not as necessary 
during the synchronizing period, and a reduction in performance can be 
tolerated. The major concern is, therefore, that the system should settle 
rapidly and stably from large signals. The effects of saturation and the 
methods of compensation for such systems are summarized in Table 8. 

Compensation for Backlash. The effects of backlash and load res- 
onance are the major limits on the performance that can be achieved with 
a power servomechanism. The great quantity of published material attests 
to the serious consideration that has been given to the problem. (See 
Refs. 13, 14, 15, 18, and 46 to 52.) However, thoroughly satisfactory 
methods for circumventing the effects of backlash are not available. 

The basic effects (see also Table 5) are illustrated by the system of Fig. 
29. For large input signals, the backlash is quickly taken up and has 
very little effect upon performance. For low-level signals approaching the 
magnitude of the backlash, the backlash tends noticeably to disconnect 
the load from the motor during signal reversals. Heuristically it can be 
seen that this will cause the load to lag farther behind the motor than with 
a linear system. Conversely because the motor is disconnected from the 
load, it will accelerate faster than normal in the backlash zone and the 
motor position will therefore tend to lead the normal response. When 
considered in terms of frequency response, if the primary feedback is from 
the load, the effect of backlash increases the lagging phase shift and de- 
creases the loop gain; if the primary feedback is from the motor, the effect 
of backlash on system performance is much less severe and it actually 
introduces a leading phase shift into the loop. For low signal levels, if 
the load damping is viscous, the linearized equations of Fig. 29 can be used. 
For low signal levels if there is appreciable Coulomb friction present, its 
effects will . predominate and the use of hysteresis to represent the back- 
lash is more correct than the equations of Fig. 29. Therefore, it is neces- 
sary to evaluate carefully the type and extent of the damping present. 
If the damping is viscous, the frequency of oscillation caused by backlash 
will generally be at or higher than the normal linear gain crossover fre- 
quency. If the damping is of the Coulomb type, the frequency of oscilla- 
tion will be lower than the gain crossover frequency. The amplitude in 
either case will be small (one to several times the backlash angle, normally). 

Methods of analysis. Phase plane, piecewise linear, and describing func- 
tion methods of analysis have been used successfully. The describing 
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w£, 2 = — — , load mechanical resonant frequency; 
Jl 

5 = , load viscous damping. 

Other constants are denned in Table 5. 

Fig. 29. Typical shunt d-c machine and load with backlash representation. The basic 
linearized transfer functions are given in a nondimensionalized form. 

function methods are the most generally useful for a paper study. How- 
ever, the complexity of the problem warrants the use of an analog com- 
puter for thorough investigations. 

As just noted and in Table 5, the representation used for backlash will 
vary depending upon the constants involved. If the hysteresis representa- 
tion is chosen, the usual describing function methods of analysis can be 
used. If the more complex representation of Fig. 29 is chosen, the method 
of equivalent coefficients, Sect. 4, is recommended. 

General Design Considerations. In the design of a power servomech- 
anism, the following basic effects should be given consideration: 

1. Use of tandem integral compensation increases the magnitude of 
sustained oscillations. The effect can be reduced by the use of dead space. 



Type of 
Corrective 
Measure 
1. Mechan- 
ical de- 
sign 



Table 9. Useful Cokrective 



Techniques for Backlash Compensation 
(a) Improved precision. The backlash can be reduced by improving the 
grade of gears used and the tolerance on the center distances, by hav- 
ing adjustable center distances, or by numerous other special design 
and/or assembly procedures. 
(6) Spring loading. There are various methods for mechanically spring 
loading the gear trains to take up the backlash. One method used in 
lightly loaded gear trains is shown at the right. This can be extended 
to the point where the entire gearing is completely divided and one- 
half spring loaded against the other half. This takes up the backlash 
throughout the entire gear train. 

(c) Split drive. By using two driving motors biased in opposite directions 
and separate gear trains for each motor, the motors will drive against 
each other to the extent of the bias and take up any backlash. A hy- 
draulic drive of this type is shown at the right. 

(d) One speed motor. Eliminate the gearing by driving the load directly 
from the motor shaft. (See Ref. 52.) 



2. Divided (a) Solid motion feedback. Consider the idealized backlash at the right, 
reset There must be a point with a displacement somewhere between the 

displacements of J m and Jl that responds only to the externally ap- 
plied forces. The displacement of this point {center of mass) is called 
the solid motion. All supplementary motions of J m and Jl relative to 
the solid motion must then be due to mutual forces that occur upon 
collision or separation and the momentum of the supplementary mo- 
tions must be equal and opposite. Instrumenting the solid motion 
would give a signal which did not contain the backlash effects. This 
point cannot be physically instrumented but by adding signals from 
the load and motor in the proper proportion the supplementary mo- 
tions can be cancelled and only the solid motion will remain. From 
the principles of conservation of momentum: 



Jm 



Jm + Jl 



%m i 



Jl 



Jl + Jm 



XL, 



where x s is the rate of change of the solid motion. (See Refs. 14, 15, 
and 49.) A method of instrumenting the technique is shown at the 
right. 
(6) Artificial damping. Reducing the load-resonant peak and increasing 
the apparent load-resonant frequency ameliorates the backlash ef- 
fects. This can be done by the proper feedback of the relative load 
and motor motions. This includes position, velocity, and acceleration 
differences. A configuration containing such a feedback is shown at 
the right. The circuit constants can be determined by frequency 
response techniques by assuming the system to be linear as in Fig. 
29. Experimental adjustment will be necessary. 



3. Network (a) Dead zone compensation. Backlash oscillations are low amplitude, 
compen- Use of a dead zone in the error channel opens the loop to low amplitude 

sation signals and will stop certain types of backlash oscillation. (See Ref. 

48.) The dead zone will be the same order of magnitude as the back- 
lash. 
(6) Frequency-sensitive networks. Compensating networks, gain changes, 
parallel-tandem networks, etc., can be used in combination with dead- 
band or separately to get the desired gain-phase change at low signal 
levels. (See Ref. 28.) Lead networks are particularly effective in 
compensating for the lagging phase shift of backlash. 
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Typical Diagrams 



Preloaded spring 



Driving shaft 




Spring tension rotates gears in opposite 
direction until backlash is taken up. 
Torque is transmitted through springs 
to shaft. 



General Remarks 

There are many other methods besides the 
two shown for mechanizing the concepts 
of (b) and (c). (See Ref. 49.) In general, 
these methods are costly and increase the 
friction and wear in the gear train. 

Electric and hydraulic models of one-speed 
motors have been built and tested. The 
low-speed high-torque requirement makes 
the electric unit bulky and heavy. 

General mechanical design considerations 
are outlined in the text. 
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U-B- 
J m = motor inertia 
J L = load inertia 



Idealized Backlash 



5 



±nt- 




When the feedback (or reset) is from (or di- 
vided between) the motor and the load, it 
is called divided reset. The concept as dis- 
cussed is highly simplified and in practice 
the configuration and constants will have 
to be adjusted to suit the particular case. 
When J m y> Jl, the motor and the center 
of mass follow closely and rate feedback 
from the motor alone is effective in damp- 
ing the system. Approximate schemes for 
obtaining the rate feedback from the mo- 
tor back emf, etc., are often adequate. 
Solid motion position feedback can be ob- 
tained in the same manner; however, this 
position signal can differ from the actual 
load position by as much as BJ m /(J m + 
Jl), and often the addition error cannot 
be tolerated. 

Feedbacks of the type in (&) are very effec- 
tive. Generally, position feedback alone 
is sensitive to system parameter varia- 
tions; rate and position feedback in com- 
bination are quite insensitive; accelera- 
tion feedback is very sensitive. Position 
difference feedback increases the resonant 
frequency, and rate difference feedback 
increases the damping. 



<-(")— Amp. — , ' — Motor — Backlash — Load 

1 



This method has been used for drives with a 
low-load inertia to motor inertia ratio (re- 
ferred to the same speed), and with suffi- 
cient friction to keep the load from much 
coasting. Under these conditions, it is 
effective and the error is small. More 
complex schemes of sensing the proper 
time to modify the gain characteristics 
are possible but usually are not justified. 
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(See Table 9, item 3.) The effects of integral type compensation achieved 
by tachometer feedback and -a lead filter have not been as thoroughly 
documented. However, the same general tendency is apparent. . 

2. Increasing the load mechanical resonant frequency, (Kl/Jl)^, reduces 
the magnitude of the sustained oscillations. 

3. It would be desirable to have mechanical load damping ratios greater 
than 0.1. These would probably be undesirable on larger drives because 
of the large power loss involved. However, there are methods of increasing 
the damping electrically (see Table 9). 

4. Primary feedback from the motor gives more stable operation than 
from the load. > 

Table 9 summarizes various methods of compensating for backlash. 

None of the schemes is perfect in the practical case, but all provide a 
certain relief from the problem. The final choice usually includes con- 
siderations of weight, size, and cost, as well as performance. 
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Fig. 30. Typical nonlinear compensating circuits. As shown the circuits vary with the 
input variable but circuits (a) and (c) can be adapted to vary with an independent 
variable: (a) nonlinear gain circuit with characteristics that vary with input voltage to 
increase gain for low-level signals; (b) resistance characteristics of the voltage sensitive 
resistor R v ; (c) nonlinear time constant circuit that reduces the time constant for large 
input signals; (d) nonlinear time constant circuit that eliminates the time constant and 
reduces the low-frequency gain for large input signals. 
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Nonlinearities to Improve System Response 

Nonlinearities used for improving system performance in general in- 
volve methods for (a) reducing the response time and/or (6) minimizing 
overshoots by more fully utilizing the performance available in the power 
element (s). Many of these methods accelerate the system rapidly for 
large errors and increase the relative damping for small errors so that 
operation is smooth (very stable). Table 10 summarizes several typical 
methods of nonlinear compensation and refers to typical circuits shown in 
Figs. 30 and 31. 



r + 



e + 



^9 




Error 



i?, 



(x) 



Nonlinearity 



(a) 



-AA/V 



+^\\-l 



■MiH 



Input to amplifier 
______ o 



R 1 C l »R 2 C 2 



=*="C 9 



=£c, 



(b) 



Feedback 



Fig. 31. Typical nonlinear feedback compensating circuits: (a) nonlinear rate feedback 
to minimize overshoot from large signals; (6) nonlinear stabilizing circuit for switching 
feedback compensation for large errors or feedback rates. To obtain the proper char- 
acteristics it may be necessary to use an isolation amplifier at (x). 



Because of the difficulty of specifying the required characteristics 
mathematically and the impracticality of instrumenting the ideal charac- 
teristics for all but the simplest systems, nonlinear compensation is ob- 
tained by empirical means in practice. 



Table 10. Typical Nonlineab Methods of Compensation 
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Description of Technique 
G m is the transfer function of the d-c output mo- 
tor, C?i is the transfer function of a convention- 
al tachometer and the dotted block represents 
the transfer function of a second tachometer in 
which the term x denotes the product of K\ and 
sc. The field of this second tachometer is ex- 
cited by the amplifier error signal so that the 
output is proportional to the error magnitude 
times the output speed. This output is sub- 
tracted from the output of the first tachometer 
and results in a value of damping which is low 
for large errors and which increases as the error 
decreases (Ref. 54). 

Generally, the relative damping is decreased and 
the frequency response increased. For in- 
stance, the solid curve represents the normal 
response for small signals, and the dotted curve 
the response for large signals. 



\ ^- Large error gain increase 




This is the same basic approach as (2), but the 
feedback allows gain and time constant changes 
to be made as functions of error and the deriva- 
tives of the error. This can be used to alleviate 
the problem of reaching zero error with high 
derivatives existing. The needed functions are 
nonlinear but can often be approximated ade- 
quately by "linear circuit components and di- 
odes. 



General Remarks 
It is possible to choose values such that this system will 
give a very fast initial response to a step input with 
no overshoot. However, if a step input in one direc- 
tion is followed by an unequal one in the opposite 
direction before the error caused by the initial step 
is corrected, the system can become unstable (Ref. 
45). This tendency toward instability can be cor- 
rected by limiting the magnitude of the term \e\'e 
to some experimentally determined maximum value. 



The increased bandwidth can be accomplished by ad- 
justing either the controller gain and/or time con- 
stants. Both methods have been used with success 
(Ref. 36). Operating on the time constants of the 
stabilizing network is particularly desirable. This 
allows the reduction of energy storage elements in 
the system which can give undesirable lags in syn- 
chronizing. Typical circuits to give gain and time 
constant change with signal level are shown in Fig. 
30. Circuit constants are determined experimen- 
tally. Since the performance of the system is de- 
pendent upon the characteristics of the inputs, one 
must completely define the input. 

Figure 31 gives two typical circuits for accomplishing 
nonlinear feedback compensation. Circuit (a) is a 
modification for a standard tachometer stabilized po- 
sition ssrvo. The form of the feedback function de- 
pends upon the servo characteristics (see Optimum 
Switching Techniques). Circuit (6) is a more elabo- 
rate feedback circuit where error and feedback rate 
are combined to switch from normal feedback to a 
feedback which provides more rapid response. Note 
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4. Optimum 
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tech- 
niques or 
minimum 
response 
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systems 
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Optimum switching is the controlled switching of 
power to the motor to reduce the error and its 
derivatives to zero in the minimum possible 
time, recognizing only the limitations on the 
performance of the motor. For example, the 
optimum response to a step input of a second 
order system with torque limiting is to acceler- 
ate at the maximum rate about halfway and 
then switch and decelerate at the maximum 
rate for the remaining distance. By proper 
selection of the switching point the system will 
arrive at zero error with zero error rate, and if 
the torque is removed, the system will remain 
at rest with no further corrective action. See 
the example in the text. Table 11 gives the op- 
timum switching functions for several second 
order systems. The number of switching points 
needed to respond in the minimum time to a 
step input is (n — 1) where n is the order of the 
system. (See Ref . 24.) Excessive switching at 
low signal levels can be avoided by having a 
small deadband at the null. Smoother opera- 
tion for small signals can be obtained by chang- 
ing the mode of operation and having a small 
linear band at null. This has been called dual 
mode operation. (See Ref. 23.) 



that for high feedback rates and low errors the switch 
will open (the diodes stop conduction) and normal 
stabilization will come into play during synchroniza- 
tion. Under extreme conditions the switch may ac- 
tually reverse polarity to allow rapid deceleration. 

It is difficult to mechanize the optimum switching func- 
tion for systems higher than the second order. How- 
ever, the optimum performance can be approached 
closely without going to the complexity of (n — 1) 
switching points. This approximation can be made 
analytically by deriving a nonlinear function (of one 
variable) that gives a response that approaches the 
optimum response. This technique is explained in 
Ref. 57. The approximation can be arrived at em- 
pirically by using the basic second order system 
switching functions and modifying them by experi- 
ence and experiment to provide satisfactory perform- 
ance for higher order systems. The optimum switch- 
ing technique is not limited to relay servos. The 
"switching" can he the saturation of some element in 
the system. In any case the optimum response is 
obtained only for the designed input; i.e., systems de- 
signed for step inputs show poorer response for veloc- 
ity inputs. Because the optimum response is the 
minimum time that a power element can make a 
correction it provides a good basis for rating system 
performance. The ratio of response time to the op- 
timum response time is a useful index of system per- 
formance. (See Ref. 55.) 
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Example. Optimum Switching Techniques to Obtain Minimum Time 
Response. (Refer to Fig. 32.) It is assumed that the amplifier gain is 
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Fig. 32. Nonlinear control system with a very high gain amplifier and torque satura- 
tion ±T m . 

sufficiently high so that the motor operates with full voltage on it for all 
but very small errors. The system equations are : 



(21) 



±T m = Jc + D6, 

m = e - /(c), 
e = r — c, 



where c = dc/dt, c = d 2 c/dt 2 , 
+ r w forw>0, 
- T m for m < 0. 



For a step input of r: 
(22) 



=FT m =J'e + De, 
m = e - /i(e). 



Equation (22) can be solved independent of time to yield a series of 
trajectories in the phase plane, Fig. 33. The coordinates of this plane are 
error, e, and error rate, e. 

There is only one trajectory which passes through the origin, and it will 
provide the optimum system response if the torque to the motor is reversed when 
this trajectory is reached. 

From Fig. 33, it is seen that proper choice of the function (e, e) — e — /(e) 
will provide the intelligence to perform the necessary switching function. 
A nonlinear tachometer feedback will then provide the necessary switching 
information. 
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Fig. 33. Phase portrait of the performance of the control system of Fig. 32, showing 
the optimum switching line where the torque must be reversed to bring the system to 
rest with no overshoot. When the quantity e — /(e) goes to zero, the torque should 

be reversed. 

Optimum Switching Functions. The form of the system character- 
istic equation will dictate what the optimum function should be. Several 
typical cases as derived in Ref. 56 are given in Table 11. 

Table 11. Typical Optimum Switching Functions for Second Order Systems 
with Limited Torque, T m 



System 
Type 

Undamped 
Viscous damped 



Torque Equation 
(See Fig. 32) 



Optimum Switching Func- 
tion in the Fourth Quadrant 



±T m =J 



dh 
dt 2 



±«w£+ fl S 



• I • i "J- m 
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1 mJ 



. _ de 
e ~Jt 



T m J . / el 



eD\ 



de 
di 



J . 
~D e > 



Coulomb damped 
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de 
dt 



° Tf(c) is positive for c > and negative f or c < and is a constant in either case. 
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1. DESCRIPTION AND DEFINITION OF SAMPLED-DATA SYSTEM 

Definition of Sampled-Data System. Systems which operate on 
data obtained at discrete intervals of time are called sampled-data systems. 
The information obtained at a particular instant is called the sample. 
Normally the intervals are equally spaced in time and the amplitude of the 
sample is proportional to the amplitude of the signal. 

Characteristics of Sampled-Data Systems 

Basic Elements. Figure 1 shows the basic elements of a sampled-data 
system: the sampler and the continuous elements. They may appear in 
various configurations, and there may be more than one sampler in the 
system. The output from the sampler is a train of pulses which is denoted 
by a starred symbol; that is, the output of a sampler whose input is e(t) is 
written e*(t). 
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Linearity. If the continuous elements are linear, the sampled-data 
system is linear and the superposition theorem is valid. A sampled-data 
system has regular time discontinuities, but the techniques of analysis 
by the use of solutions of the linear constant-coefficient differential equa- 
tions of the system are directly applicable. 
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Fig. 1. Sampled-data system and sampler input and output signals: (a) simple sampled- 
data system; (6) continuous error function; and (c) sampled error function. 



The Sampler. The sampler acts as a pulse modulator of the input and 
generates a train of pulses. This action introduces high frequencies into 
the system which may be attenuated by a linear filter. The information 
contained in the input signal may be recovered with reasonable fidelity if the 
sampling frequency is at least twice the highest frequency component of the 
input signal. Figure 2 shows the effect of sampling frequency upon the 
frequency spectrum of the output of the sampler. 

Use of Samplers. Sampled-data systems may be used for several 
reasons : 

1. To use a digital computer as part of the controller. The input data 
must be in sampled form. 

2. To use simpler, low-powered control elements. 

3. To realize the beneficial effect which sometimes accrues when sampled- 
data systems are used for the process control of plants having inherent 
dead time. 
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Fig. 2. Sampler transfer characteristics in the frequency domain, (a) Amplitude 
spectrum of sampler input; (b) amplitude spectrum of sampler output, sampling fre- 
quency greater than twice the maximum signal frequency; (c) amplitude spectrum of 
sampler output, sampling frequency less than twice the maximum signal frequency. 
w s = sampling frequency (Ref. 5). Reprinted by permission from J. D. Truxal, Auto- 
matic Feedback Control Systems Synthesis, Copyright 1955 by McGraw-Hill Book Co. 

4. To use pulsed-data information. The input information may be 
available in discrete samples as in a guided missile control system or as in 
certain track- while-scan radar systems. Sampled-data systems may be used 
to advantage where digital sensors are already available. 

Description of Typical Sampled-Data Systems 

Digital Computer in the Controller. Figure 3 shows a typical digital 
system. The sampling and coding unit converts continuous data into 
pulsed data. The digital computer performs a series of operations on the 
pulsed data and presents the results in pulsed form to the holding and 
decoding unit which reconverts the results into (approximately) continuous 
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signals for use by the continuous control equipment. The feedback may 
transmit the data in either pulsed or continuous form. 
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Fig. 3. Typical sampled-data control system. 

continuous. 



Conventional control equipment is 



In practical operating systems, a typical method of converting from a 
continuous variable available as a shaft rotation to a binary code number 
which represents its magnitude and polarity is to use an encoding device 
such as a circular binary pattern shown in Fig. 4. The circular tracks may 




Fig. 4. Circular binary pattern for analog-digital conversion. The lines across the 

pattern show that accurate angular position of the photocells or brush contacts is 

necessary to avoid errors in conversion. 

be scanned radially with a photoelectric cell or brush pickoffs. The output 
will be the binary pulse code which represents a particular position of the 
circular binary pattern; the pattern shown can resolve a circle into 2 6 = 64 
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parts. Although the encoder shown is for angular rotation, devices have 
been manufactured for conversion of pressures and flows to digital form. 
Techniques for converting analog voltages to digital form are also available. 
(See Vol. 2, Chap. 20.) 

Periodic Process Controller. A typical sampled-data regulator for 
process control is shown in Fig. 5. 




Periodic switch 

closed when 

t = nT 

Fig. 5. Typical sampled-data process regulator. 

The typical stepwise process controller monitors the controlled variable 
periodically (every nT) and makes a control adjustment at each sensing 
instant. In process regulation, the usual (and perhaps the most useful) 
form of control actuator is a servo motor, which serves as a low-pass filter 
and also serves to reset the error detector. The following description of a 
periodic controller is taken from Oldenbourg and Sartorius (Ref. 1). 




D-C servo motor 



To process control valve 
Fig. 6. Schematic form of a periodic controller (chopper bar relay) (Ref. 1). 
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From a constructional standpoint, the periodic controller operates about 
as follows. Through a sensing device, such as a meter pointer, the control 
variable is observed at equal time intervals. Then, by auxiliary power, 
additional members of the control loop are suitably actuated according to 
the sensed position of the pointer. 

Example. The Chopper Bar Controller. See Fig. 6. As long as the 
meter pointer stands between the two contact springs the circuit remains 
broken, even during the sensing instants. It is closed only when the pointer 
leaves its mid-position. The duration of closure increases with deviation 
of the pointer. If the contact closure is used to actuate a reversible 
constant-speed motor, the control action is called astatic (never quiet) 
because, with constant actuating error, the control motor moves inter- 
mittently across its entire range at an average speed (roughly) proportional 
to the pointer deviation. Although periodic controllers may have static 
correspondence between deviation and motor motion, astatic action will 
be assumed here because of its greater practical significance (Ref. 1). 

2. METHODS OF TRANSIENT ANALYSIS 

Basic Mathematical Relationships 

Analysis of Sampler. The output of the sampler (see Fig. 7) is the in- 
put modulated by the sampler into a train of pulses: 

00 00 

(1) e*(t) = c(0 X) u (t - nT) = £ e(nT)u (t - nT), 

n=0 n=0 

where u (t — nT) = the impulse or Dirac delta function occurring at 
t — nT, in which 

a u(t — nT) - u(t - nT - a) 

Uq{1 — nT) = lim > 

a->o a 

T = the sampling period, 

n = an integer, 

e{nT) = the value of the input at the sampling instant. 

The Laplace transform of eq. (1) may be written: 

oo 

(2) E*(s) = X) e(nT)e- nTs , 

n=0 

or eq. (1) may be written in the frequency response form: 

(3) E*(s) =i E E(s + jn2ir/T). 

■* n=— oo 
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c*(t) 



e(t) 



o \ o _e*fO_ 



8(0 



c(t) 



Fig. 7. Showing basic mathematical relationships of a sampler. 

Note. Equation (3) may be derived by performing the complex con- 
volution of the input e(t) and the train of unit impulses generated by the 
sampler, namely 



(4) 



E*(s) = E(s)®£ 



£ u (t - nT) 



where ® is the symbol denoting complex convolution. 



Notice that 



2 Mt ~ nT) 

1-71=0 



= 1 + e~ sl + e 



sT I .— 2sT 



+ 



or in closed form: 



Because 



1 



[1 - exp (-sT)] 
has only simple poles at s = jn2w/T, the complex 



[1 - exp {-sT)] 
convolution reduces to eq. (3). 

Smoothing the Sampled Data. Normally, the high-frequency com- 
ponents generated by the sampler are removed before the signal reaches 
the output. Often in sampled-data servo systems, a large portion of the 
smoothing is accomplished by the components (motors, etc.) between 
the sampler and the output. Sometimes more smoothing is necessary. 
One particularly simple low-pass filter is the holding circuit or 'boxcar 
generator. In this circuit, the value of a sampling pulse is held until the 
next pulse arrives, whereupon the circuit assumes the value of the new 
pulse. The transfer function of such a network is that of a rectangular 
pulse of unity height and of T seconds duration, namely 



(5) 



Gh(s) =-[l-e 
s 



-sTi 



Response of a Continuous Filter to Sampled Data. The response 
of a continuous transfer member g(t) of Fig. 7 is 



(6) 



c(t) = £ g{t)e(nT)u Q (t - nT), 

n=0 
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Equation (6) has the Laplace transform 

(7) C(s) = E*(s)G(s). 

Equation (6) is a summation of the filter impulse responses which are 
excited by each sample and is valid only for the case of zero initial condi- 
tions. When this condition is not met, a second term must be added to 
eq. (6) to include the decay of the nonzero initial conditions. Since the 
system is linear this is not important when considering the stability of the 
system, but it must be included if a time response is being calculated. 
The response of the filter only at the sampling instants is: 

CO 

(8) c*(t) = E e(nT)u (t - nT)g(t)u [t - (q - n)T], 

n=0 

which has the following Laplace transform : 

(9) C*(s) = E*{s)G*{s), 

where 

G*(s) = £[g*(t)] = £[g(t)u (t - nT)}. 

Sampled-Data System Transfer Function. From eq. (9), the 
sampled-data transfer function or pulse transfer can be defined as 

C*(s) 

( 10 ) £*(*) = ^TV 

E*(s) 

An equivalent form in terms of the ^-transform symbolism is indicated 
in eq. (11). (The 2-transform is defined and illustrated in a later paragraph.) 

C{z) 

(11) G ^ = TT,' 

E{z) 

Laplace Transform Analysis 

It is possible to use the equations of the previous section to obtain the 
complete time response, c(t). However, the Laplace transforms are not 
rational and it requires considerable labor to obtain the complete response. 
If the response is calculated only at the sampling instants, the transforms 
can often be written in closed form and the labor of calculation and manipu- 
lation is greatly reduced. The ^-transform method is usually used to com- 
pute the response at the sensing instants. (See ^-Transform Analysis, 
Sect. 2.) 
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Analysis by Difference Equations 

The analysis of sampled-data control systems leads to characteristic 
equations, which are difference equations. 

Formulation of the Difference Equations. Difference equations are 
discussed in Chap. 4. A simple example will illustrate the analysis of a 
control process by difference equations. 

Example. (See Fig. 8.) The following simplifying assumptions are 
made: (a) The displacement of the control means, w, is linear and unlimited. 



me 



Supply 



Control valve 



Process 



Influx 



1/OTcS + l) 



Controller 



K/s 



e(nT) 
(a) 



c(t) 



>^ 




2J< — r o 
e(t) 




(b) 

Fig. 8. (a) Simple process control. (6) Elements of process control equivalent to (a). 
Note. e(t) = ro — c(t) ; this quantity is dimensionless. 



(b) The plant has a simple time constant, T c . (c) The controller is lag- 
free, i.e., the sensing and positioning times are negligible. 

The disturbance is assumed at the most unfavorable instant (just after 
sensing). The control means, m, changes instantly at the sensing instant 
and remains at its new value throughout the sensing cycle. 

The analysis is quite simple. Inside a sensing cycle, the behavior of 
the plant is continuous and may be described by the linear differential 
equation 

T c de(t) 

— — h e(t) = r — nii + m e = r — m 

1 U,T 
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where T c = the plant time constant, seconds; 
T = the sampling cycle, seconds; 
e(t) = the controlled variable error, dimensionless: 

e(t) = r — c(t), c(t) is the controlled variable, r is the set 
point, all parameters nondimensional and normalized; 
t = t/T, dimensionless time; 

m = the net value of control means, dimensionless; 
irn = the manipulated control means, dimensionless; 
m e = the disturbance of the control means, dimensionless. 

By using the Laplace transformation it can be shown that the solution 
to eq. (1) at the nth sensing instant is 

(12) e n = De n _ x - (1 — D)m n _i, 

where e n = value of the controlled variable at the nth sensing instant, 
e n _i = value at the (n — l)th sensing instant, 

D = exp (— T/T c ), the decrement characteristic of the plant and of 
the sensing cycle, 
m n _i = value of the control means at the (n— 1) sensing instant. 

Now consider the behavior of the variable m at the sensing instants. 
The relationship is assumed to be linear: 

(13) m n — m n _i = Ke n , 

where K is the strength of the controller and is called the specific step. 
The minus sign in eq. (13) provides the negative feedback needed for 
regulation of the variables. 

Equations (12) and (13) are the simultaneous difference equations of 
the control action. They lead to the difference equation of the system, 
namely, 

(14) e n+2 ~[1 + D-K(l- D)]e n+1 + De n = 0. 

Solution of Linear Difference Equations. The linear homogeneous 
difference equation may be written : 

(15) A e n+q + Aie n+q _i H h A q _ie n+1 + A q e n = 0. 

If the roots are not equal, eq. (15) has the solution: 

Q 

(16) e n = 2 a i z i n > 

where zi is a root of the auxiliary equation, 

(17) A Q z q + A lZ *- 1 + • • • + A q _ lZ + A q = 0. 
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The coefficients of eq. (17) are identical with those of eq. (15). Equation 

(17) is often called the characteristic equation of the system. 
With q distinct roots the general solution is 

(18) e n = .a x z{* + a 2 z 2 n + a 3 z 3 n -\ \- a q z q n . 

If k roots are equal the solution is 

(19) e n = [oi + a 2 n -\ h a^ -1 ]^ -f- a k+1 z 2 n H h a q z q _ k n . 

Thus, the values e n at any sensing instant may be computed. The q 
summing constants a* are determined by the first q of the e's at the sensing 
instants. The characteristic equation with vanishing roots has special 
significance as will be discussed later. 

Use and Limitations of Difference Equation Method. If the prob- 
lem is dominated by the sampler, that is, if there is a rather simple control 
loop whose servomotor is actuated by a periodically applied measurement 
of the error, this type of analysis is simplest. It is also well to remember 
that the difference equation method and the 2-transform method are 
synonymous. For higher order systems, the difference equation approach 
becomes laborious, so that the more methodical 2-transform method be- 
comes advantageous. 

Analysis by s-Transform Method 

Usefulness. The 2-transform is the shorthand rational way to write 
the Laplace transform of the linear difference equation. It has the same 
relationship to linear difference equations as the Laplace transformation 
bears to linear differential equations. The advantages of the ^-transform 
are: (a) it reduces the nonrational Laplace transform of a sampled-data 
system to a rational transform which facilitates writing transfer functions; 
(6) it allows definition of the closed loop system response of a sampled- 
data system (no advantage over difference equations). 

Limitations. Tables of the more complex 2-transforms are not readily 
available and the polynomials must be expanded into partial fractions. 
A fundamental limitation of ^-transforms is that the time solutions are de- 
fined only at the sensing instants. 

Hidden Oscillations. Because the time solutions are calculated only 
at sensing instants, it is possible that the sampling frequency may be 
lower than the characteristic frequency of the plant being controlled, and 
oscillations may occur which are not apparent from the ^-transforms. If 
such a condition is suspected the 2-transform can be modified to give the 
output between the sampling instances and the existences of such oscilla- 
tions can be checked. (See Refs. 4 and 5.) 
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Basic Relationship. The ^-transform is based on the transformation 

(20) z = e sT , 

where s is the Laplace operator and T is the sampling period. The Laplace 
transform of the sampled signal will contain s in the irrational form 
e~ nsT . Substitution of z will produce a rational transform in z. The z- 



transform is defined as 


CO 






(21) 




c(z) = E 

rc=0 


c(nT)z- n . 




Table of Useful z -Transforms. ! 


See Table 1. 






Table 1. Laplace and z- 


-Transforms (Refs. 2, 


,5) 




Column 1 


Column 2 
Time 


Column 3 


Column 4 
Description of 


Row 


Laplace Transform 


Function 


z-Transform 


Time Function 


a 


1 


u (t) 


1 


Impulse function 

aU = 


b 


e -nTs 
1 


u (t - nT) 


1 


Impulse function 
at t = nT 

Train of impulses 




i(t) 


Z 


at sampling in- 
stants 


c 


1 - e- Ta 


z - 1 


d 


1 

s 


u(t) 


z 


Step function 


z - 1 




1 

o 
S~ 


t 


Tz 


Ramp function 


6 


(z - D 2 




1 

7* 




i T2 z( z + D 
2 (z-1? 


Quadratic or ac- 


f 


1/2 


celeration func- 
tion 




l 

s + a 


e -at 


z 


Exponential func- 


9 


z - e- aT 


tion 


h 


a 


sin at 


z sin aT 


Sinusoidal func- 


s 2 + a? 


z 2 - 2z cos aT +1 


tion 




1 


a^ 


z 


Constant raised 


% 


s - (1/T) In a 


z — a 


to power t 




b 


a t,T sin bt 


za sin bT 


Sine wave multi- 


3 


[s - (l/T) In a] 2 + b 2 


z 2 - 2az cos bT + a 2 


plied by a t!T 


k 


s - (l/T) In a 


a t,T cos bt 


z(z — a cos bT) 


Cosine wave mul- 


[s - (l/T) In a] 2 + 6 2 


z 2 - 2az cos bT + a 2 


tiplied by a t/T 




F(s + a) 


e~ at f(t) 


F(e +aT z) 


Effect of multipli- 
cation by e~ at 
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Methods of Inverting ^-Transformation. 

Real Inversion Integral. The real inversion integral for the z-transform is 



(22) 



c(nT) = — (f)C(z)z n - 1 dz, 
2irj J 



where the line integration is made of a sufficiently large radius to enclose 
all roots of the integrand. 

Partial Fraction Expansion. The z-transform is factored into com- 
ponents so that each term in the expansion can be obtained from Table 1. 
The usual methods for partial fraction expansion are applicable. (See 
Chap. 20.) 

Power Series Expansion. From eq. (21) 

" c(0) c(T) c(2T) 

(23) C(z) = £ c(nT)z-» = -V + ~V + "V" + ' ' '> 



which thus expands the z-transform of a variable in an inverse power series 
in z. • The coefficient c(nT) of z~ n is the value of the variable at the nth 
sensing instant, and the coefficients can be used directly to plot the time 
function at the sampling instants. 

^-Transform Block Diagram Algebra. The z-transform describes 
the transfer function of two variables at the sensing instants only. Figure 
9a illustrates the transform R{z). Figure 96 shows the transform 



(24) 

and Fig. 9c 

(25) 

(a) 
(b) 

(c) 

(d) 



r(0 



r(t)^ ^ r*(t) 

>— cr o 



C b (z) = R(z)G 1 (z), 
C c {z) = R{z)G l {z)G 2 {z). 

r*(t) 
=»— 

/ c b *(t) 



c*(t) 



r(t) 



^r*(t 



lOL^S'C 




Fig. 9. Basic z-transform relationships. 
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In words, if each transfer member is separated from others by synchronous 
samplers, the ^-transforms cascade, i.e., they can be multiplied. But 
notice that if the transfer members are not separated by a chopper, the z- 
transform cannot be obtained by multiplying together the ^-transforms 
of the component members. In continuous systems where coupling exists 
between transfer members a similar difficulty is encountered. For ex- 
ample Fig. 9d has the transform : 



Consider 



COO = R(z)G 1 G 2 (z). 

Gi(s) = — !— ; GxCO = 



s+ 1 z - exp(-T) 



1 
and G 2 (s) = — ; G 2 (z) = 



Then 



whereas 



s + 2 z - exp (-2T) 

Cc(z) = _^ ^ 

R(z) [z - exp(-T)][z - exp (-2T 7 )]' 

C d (z) z z 



Note. 



R(z) , [z - exp (-T)] [z - exp (-27 1 )] 

z[exp (-T) - exp(-2T)] 
• ~ [z - exp (-T)][z - exp (-2T)] ' 

G 1 (z)G 2 (z) * G 1 G 2 (z). 



The difference is that in the first case, G 2 is driven by a train of impulses, 
whereas in the second, it is driven by the linear response of Gi to its own 
input pulses. A helpful concept is that the ^-transform of a chain of transfer 
members must be derived from chopper to chopper in the circuit. Table 
2 shows some control loops, their Laplace transforms and their ^-transforms. 
The output c may be assumed to be sampled by an imaginary chopper 
(synchronized with the real one), resulting in c(nT), although this im- 
aginary chopper must be disregarded in traversing the complete control 
loops. 
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Table 2. Output Transforms for Basic Sampled-Data Systems 



System 

T —*S« 



^O^ 



■H- 
H- 




^o 



Laplace Transform 
of Output C(s) 


z-Transform 

of Output 

C(z) 


R*(s) 


R(z) 


GR*(s) 


GR{z) 


G(s)R*(s) 


G{z)R{z) 


G(s)R*(s) 
1 + HG*(s) 


G(z)R(z) 
1 + HG(z) 


G*(s)R*(s) 
1 + H*(s)G*(s) 


G{z)R{z) 
1 + H(z)G(z) 


r«.i HmG ' (s) -\ 


RG(z) 



G 2 



+ 



G 2 (s)RG 1 *(s) 
1 + //(?i(? 2 *(s) 



*(«)■ 



1 + HG{z) 



G 2 (z)RG 1 (z) 
1 + HG,G 2 (z) 



a This table is reprinted from an article by Ragazzini and Zadeh (Ref. 2) with the 
permission of the authors. 

3. SAMPLED-DATA SYSTEM STABILITY 

Stability Criteria of Difference Equations and ^-Transforms 

The solution of the characteristic difference equation with nonequal 
roots is 

(26) 



c n = c(nT) = 2 a i z i n - 



z=i 



Now if c{nT) is to remain finite even for large n, then 

(27) zi 1 < 1 or |z,-|<l. 

In words, the inequality (27) states that, for stability of the sampled-data 
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system, the roots (or zeros) of its characteristic difference equation must 

lie inside the unit circle with center at the origin. The unit circle in the 

z-plane is the periodic limit of stability corresponding to the Routh-Hurwitz 

stability criteria. 

The Routh-Hurwitz Stability Criteria. This is used in linear control 

theory and may be applied to sampled-data systems by using the conformal 

transformation 

2 + 1 
(28) s = -• 

2—1 

This transformation changes the unit circle in the 2-plane into the left half 
of the s-plane as shown in Fig. 10. If the Hurwitz conditions are applied 




Fig. 10. The linear transformation s w = (z -f- l)/(z — 1), used for deriving stability 
conditions from the difference equation of control. 

to the characteristic equation (subjected to the transformation of eq. 28), 
the conditions can be found which cause the roots of the transformed 
equation to lie in the left half of the w-plane. Hence, the roots of the 
characteristic equation must lie within the unit circle in the 2-plane. As 
an example consider the second order characteristic equation 



(29) 



A z 2 + A x z + A 2 = 0. 



If the transformation (eq. 28) is used, the transformed equation is 

(30) B s 2 + B lS + B 2 = 0, 

where J5 = A + A x + A 2 , 
B x = (A - A 2 ), 
B 2 = A - A x + A 2 . 

In the simple quadratic case, the Hurwitz criterion requires for stability 
only that all these coefficients have the same sign. Therefore, the periodic 
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stability limit of the second order eq. (29) is defined by the dual condition 

(ifBo>0): 

5i > or A - A 2 > 0, 
(31) 

B 2 > or A - Ai + A 2 > 0. 

The above procedure can be extended to higher order systems. 

Use of Frequency Response Methods to Determine Stability 

Nyquist Diagram. Exact Graphical Procedure. The Nyquist diagram 
can be drawn by considering G(z) rather than G*(s). The complex plane 
plot is made by allowing z to vary along a unit circle in the z-domain. The 
gains and phase at any frequency are found by locating the point on the 
unit circle (z-domain) corresponding to this angular frequency. The inter- 
pretation of the Nyquist diagram follows conventional lines. 

Example. Simple sampled-data system {T = sampling period) (this 
example is after a similar example by Truxal, Ref . 5) : 



K 

G(s) = = K 



1 



G*(s) = 



s(s+l) Is s+U 

z z 



Kz(l - e- 1 ) 

K = — 

U - 1 (z - e~ T )\ (z - l)(z - e~ T ) 



for co s = 4 rad/sec, 

2t 6.28 



— =T = = 1.57; e -1 - 57 = 0.208, 

w s 4 

0.792ifz 
G*(s) = 



(z - l)(z - 0.208) 



The unit circle in the z-plane would appear as in Fig. 11. The vectors are 
shown for 1 rad/sec. At <o = 1 rad/sec, 



G*(jl) = 



z = 1 /90° , 
0.792iH/90 c 



1.414 /135° 1.0216 /101.8 C 



likewise 



K 

= 0.86 — /- 146.8° ; 

rp i ' 

K 
G*(j2) = 0.52- /180° . 
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Imaginary z 
1 rad/sec 




Unit 
circle 



Fig. 11. Pole-zero configuration for G(z), with vectors shown for calculation of Nyquist 
diagram at w = 1 rad/sec 

<*.)- K 



and w s = 4 rad/sec. 



s(s + 1) 



Continuing the above for other frequencies, a Nyquist plot, G*(ja>), si- 
milar to that shown in Fig. 12, could be produced. 

Comparison with Amplitude Modulation. Graphical Approxi- 
mate Nyquist Plot. Linvill applied the Nyquist diagram to sampled 
systems based on an approximation for the starred open loop transfer 
function : 

G*00 =i E G(s+jnu s ). 

J- n= — oo 

If G(s) is a good low-pass filter, G*(s) will contain only two or three signi- 
ficant terms. For example G*(jl) is the vector addition 

G*(jl) = (l/T)[G(jl) + G(jl - M + G(jl + i«.) +-..]. • 

If the sampling frequency is 4 rad/sec, 

G*(jl) = [G(jl) + G(-j3) + G(j5) + G(-j7) + • • -](1/T). 
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240 
120 



230 
130 



220 
140 




120 
240 



110 

250 100 
260 



Fig. 12. Nyquist diagram for G*(s) with <?(«) = K/s(s + 1), K/T = 1, and w s = 4 

rad/sec (constructed by using two-term approximation) (Ref. 5). Radial scale numbers 

are in terms of T/K; circle spacing is Q.125T/K. 



All terms except G(jl) and G(—jd) would be small if G(s) is an effective 
low-pass filter. 

The graphical construction on the G*(ju>) plane is shown in Fig. 12. 

Note. The value of G*(ju s /2) is purely real. At frequencies above 
co s /2 the Nyquist diagram continues into the upper half-plane until it 
reaches infinity at the sampling frequency. The only part of the diagram 
of interest in stability considerations is the section corresponding to fre- 
quencies lying between zero and co«/2. 

The example illustrates that sampling, by itself, increases the phase 
lag for a given gain. From the Nyquist diagram the maximum gain for a 
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stable system is read directly. If only the two terms are used in the series 
expansion of G*{ju>) the allowable K/T is 2.5, if all are used, the K/T is 
1.94. The gain in the first case is 3.93; consideration of the rest of the 
terms reduces the allowable gain to 3.05, since in this example, T = 2-71-/4 
= 1.57. 

4. SAMPLED-DATA SYSTEM SYNTHESIS 

Design Procedure Using ^-Transforms 

This section is based upon material from Ref. 4. 

The typical sampled-data system of Fig. 13 will be used for illustration. 
The error unit embodies both the analog-digital transducer, which peri- 



Input 



Error unit 



Subtractor 



Digital to 

analog 

converter 



Transducer 



L. 



Controller 
G c (z) 



Motor- 
amplifier 
G m (z) 



Output 



Fig. 13. The basic digital servo system. 

odically expresses the angular position of the output shaft as a number in 
binary code form and the digital subtractor which takes the difference of 
this number and the incoming one. The characteristics of the servo motor- 
amplifier combination differ for different applications. They are assumed 
to be known and invariant, so that the problem is to synthesize a suitable 
controller. The following ^-transforms will be used: 

G c (z) = 2-transform of the controller, 

G m {z) = z-transform of the motor-amplifier combination, 

G (z) = 2-transform of the open control loop, 

G(z) = the closed loop z-transform. 

z-transform of the forward path 



(32) 



G(z) = 



1 + Go(z) 



Performance Criteria. It is useful to assess the performance in terms 
of the responses to specific driving functions such as a step function, a 
steady velocity or acceleration, a sinusoidal input of various frequencies or 
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a random noise. Any or all such tests may be applied and, since improve- 
ment in one respect is often accompanied by deterioration in another, it 
will be necessary to compromise. Such overriding factors as the demand 
for zero velocity lag must take precedence. The servo amplifier may over- 
load if it is fed a series of discontinuous pulses representing samples. Its 
input must be reasonably smooth, and the correction due to one error 
number will not be complete before the next is begun. 

An equivalent system is shown in Fig. 14. The system delay, XT', is 
now shown with the motor amplifier. The controller has two parts, the 
first of which is characterized by its 2-transform, G\(z), and it modifies the 
sequence of correction samples supplied to it. The modified sequence is 



Gi(z)- 



-G 2 G m (z) 



Input 

— *— 



-i I 

I ! 



I i 

I L. 

• Controller- 



G 2 (s) 



Motor- 
amplifier 
G m (s) 



G (z) 



Delay 

W- 



j Output 



xr 



Fig. 14. A system equivalent to that of Fig. 13. 



smoothed by the second part, characterized by its transfer function G 2 (s), 
which provides a continuous signal for driving the servo amplifier. This 
subdivision is unlikely to correspond to any physical separation of the 
components. The composite expression G\(z) * G 2 (s) may be called the 
"operational instruction" of the controller. The * symbol is used in this 
case to separate the sampled and continuous portions of the operational 
instruction and indicates that the information input to the continuous 
elements is in sampled form. 

Knowing G m (s) and the performance requirements, it should be possible 
to specify G 2 (s). For example, the motor amplifier may have the simple 
transfer function l/[s(l + T m s)], where the time constant, T m , is probably 
smaller than the sampling interval. To avoid sudden changes in velocity, 
G 2 (s) need only be 1/s. 

The next step is the determination of a suitable Go(z), taking into account 
all the overriding factors. The fact that G (z) may be expressed as the 
ratio of two polynomials N(z)/D(z) is also used. 

Physical Realizability. The order of N must be at least one less than 
that of D. 

Poles at z = 1. To have zero static error, the function G (z) must 
possess at least one simple pole at z = 1. A second order pole at z = 1 
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would provide zero velocity lag and a third order pole, a zero acceleration 
lag characteristic. 

Cancellation of Poles and Zeros. The characteristic equation of the 
system is D(z) + N(z, A) = 0. The parameter A indicates the need for 
checking the values of the system variables between sensing instants. The 
system will be unstable if any root lies on or outside the unit circle (z) = 1. 
One may be tempted to arrange by adjustment of parameters for the can- 
cellation of a zero by a pole so as to eliminate the root which would other- 
wise lie outside the unit circle. It is better to increase the sampling fre- 
quency. This point cannot be emphasized too strongly, particularly be- 
cause it is tempting to deal with the special case of no system delay, but 
this can result in instability that would then be revealed only when the 
behavior between sampling instants is investigated. See Ref. 5. 

Design Constants. The suggested method of synthesizing a system is 
to match the characteristic equation with one known to give satisfactory per- 
formance. Lawden et al. (Ref. 6) has used equations of the form (z — a) n = 
0, although when n is a small number, it may be desirable to depart from this 
form. Oldenbourg and Sartorius (Ref. 1) show that minimum control 
area (see Condition for Minimal Control Area, later in this section) results 
from the case of vanishing roots, namely z n = 0. Examples relevant to 
continuous systems may well be suitable for sampling systems. The 
procedure is to arrange for N(z) and D(z) to include between them a number 
of constants which are adjustable in the design stage. This number should 
be equal to the order of the characteristic equation. It is always possible to 
do this, because two additional constants are picked up each time the order 
of the characteristic equation is increased by one. The characteristic 
equation z n = leads to minimum control area, but if there is noise present, 
very little smoothing is provided; as a result, the servo amplifier may be 
transiently overloaded or driven into saturation. The characteristic equa- 
tion (z — 0.4) n has been used by some authors to provide smooth and 
satisfactory performance in the presence of noise. Analysis of a representa- 
tive second order sampled-data system by Jury (Ref. 7) leads to the results 
of Fig. 15, which shows the constant overshoot loci in the z-plane. It can 
be shown that these loci can be used for higher order systems. Note that 
a system which has no overshoot has its characteristic roots on the positive 
real axis. The values of the roots must be less than unity. 

The simplest expression which meets all the above requirements is the 
expression G (z). Dividing it by G 2 Gm(z) gives G\(z), the first part of the 
operational instruction. 

The Operational Instruction. It remains to decide how standard 
components may be assembled into a system having the required opera- 
tional instruction, G\{z) * G 2 (s). The s-part must describe the properties 
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Fig. 15. 



>-Real 



Constant overshoot loci in the z-plane for a unit step input, M p = ratio of 
values transient peak to steady state (Ref. 7). 



of the digital-analog converter included in the error unit. This converter 
may perform the function of a clamp, which has the operational instruction 
(1 — e~ sT )s~ 1 . Other functions of s may be obtained by the usual syn- 
thesis procedures and may lead to further terms in the 2-part. This usually 
leaves an expression in z which is required for the rest of the operational 
instruction. Generally, such an expression is of the form 



(33) 



A + A lZ ~ l + A 2 z~ 2 ^ 1- A r z 

1 + B lZ - 1 + B 2 z~ 2 + • • • + B r z~ 



This function may be synthesized in many ways; one way of constructing 
its physical counterpart is with the aid of r delay elements (each equal to T). 
The output is obtained by the summation of delayed components pro- 
portional to the coefficients in the numerator. The correct denominator is 
obtained by negative feedback of the delayed components proportional to 
the coefficients in the denominator. 

Example. Synthesis of a Simple Analog System. Figure 14 shows the 
simple system to be synthesized, and the following assumptions are made 
regarding it: 

(a) The servo motor and amplifier are constructed so that the rate of 
rotation of the motor is proportional to the voltage applied to the amplifier. 
The transfer function is G m (s) = K/s. 
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(6) The transducer samples and introduces a delay, T, the effect of 
which is to multiply the z-transform by z -1 . 

(c) There must be zero static error; hence G (z) must include the 
factor (z — 1) in its denominator. 

(d) There shall be no sudden changes in output velocity. The law of 
motion shall be quadratic between sampling instants. Hence G 2 (s) = s~ 2 . 
Therefore, G 2 (s)G m (s) = Ks~ 3 and G 2 G m (z) = KT 2 z(z + l)/2(z - l) 3 , 
the z-transform being found directly from Table 1. 

The z -Transform of the Loop. This is 

KT 2 G l {z){z+ 1) 

(34) G (z) = , '« * 

It contains the required factor (2 — 1) in the denominator. It must also 
contain adjustable design constants such that the characteristic equation 
can be forced into one known to be suitable such as (2 — a) n = 0. The 
simplest expression for G\(z) which adds two further constants without 
increasing the order is 

(35) (?i(«) = (z - l) 2 /(2 2 + B lZ + B 2 ). 
The operational instruction for the controller is therefore 

(z ~ I) 2 



(36) 



(z 2 + B x z + B 2 )*s 



—2 



The 2-transform of the operational instruction is obtained by replacing 
s~ 2 with its z-transform, Tz(z — l) -2 , which reduces eq. (36) to Tz{z 2 -\- 
BiZ + B 2 )~ l . The characteristic equation is 

(37) z 3 + (Bi - 1)2 2 + (B 2 -Bi + l/2KT 2 )z + (1/2KT 2 - B 2 ) = 0. 

The simplest third order equation this can be identified with is z 3 = 0. 
Choice of the characteristic equation z 3 = is known to produce the most 
rapid recovery from a transient disturbance. (See Condition for Minimal 
Control Area, later in this section.) If KT 2 = Bi = 1 and B 2 = %, G (z) 
can be written 

(38) G (z) = (z+ l)(2z 3 - z - I)" 1 . 
The closed loop z-transform in response to a pulse is 

(39) G(z) = Go(z)[G (z) + l]-\ 

or 

G(z) = \z~ 2 + \z~\ 
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Design Procedure Using Frequency Response. The design proce- 
dures of linear techniques discussed in previous chapters are applicable. 
Nyquist and Bode diagrams may be used after the transfer function has 
been obtained. In working with the Nyquist diagram, it can be seen that 
lead compensation will increase the bandwidth of the system. If the 
sampling frequency is not increased, no additional high-frequency informa- 
tion will be passed. This illustrates a difficulty which one may encounter 
in the synthesis of sampled data systems. 

Design Procedure Using Root Locus Techniques. As in frequency 
response techniques, the root locus could be used as an aid for synthesizing 
sampled-data systems. However, it can be shown that the desired root 
locus is the positive real axis in the 2-plane. As previously mentioned, the 
characteristic equation z n = is known to lead to the fastest recovery of 
the system from a disturbance. If noise is present {z — a) n = is the 
desired characteristic, where a is a number between zero and 1. As Jury 
has shown (see Fig. 15), the loci of constant overshoot also illustrate that 
the positive z-axis is the desired place to locate the roots of the characteristic 
equation. The circumference of the circle in the z-plane having unity 
radius is the periodic limit of stability. 

In summary, the sampling frequency controls the bandpass and thus 
the speed with which the system can transmit information. The roots of 
the characteristic equation should be placed as near the origin as permissible. 
Placement at the origin is known to produce the liveliest system; if noise 
is present, the roots must be moved along the positive real axis in the 
z-plane toward 2 = 1. It should be noted that in many practical cases 
the above simple criterion for performance will have to be modified for 
one or more practical reasons. In such cases the approach suggested is 
to use the above rules for the first approximation and then to introduce 
the other considerations. 

Performance Charts for Typical Sampled-Data Systems 

Performance Index: Control Area. To evaluate the results of com- 
putations and to choose the most favorable conditions of operation, the 
concept of control effectiveness, measured by the smallness of the control 
area, is very useful. For continuous controllers, the control area is de- 
fined as 

(40) F = I e(t) dt 



-/ 



For sampled-data controllers the calculation is not so simple, except in 
one case, when the control process has the initial value of zero at the first 
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sensing instant. In this case (which leads to the largest control area) the 
control area is 

F °° 

(41) — - = 23 e n, T = sampling period. 

T n =0 

It can be seen the control area is the error-time integral. 

Condition for Minimal Control Area. The characteristic equation 
with vanishing roots, namely z n = 0, has the least control area. Such a 
system can be shown to recover most quickly from a disturbance. How- 
ever, if there is noise present, or if such a characteristic equation is physi- 
cally unrealizable, the characteristic equation (z — a) n = is used. It 
is used in the presence of noise to provide smoothing of the impulses, and 
it is used in the second case so that normal system components may be 
employed. 

Second Order System with Dead Time and with No Compensa- 
tion. (See Fig. 16.) The characteristic equation for this system is 

(42) z 2 4- [K(l - D/L) - (1 + D)]z + D - KD(l - 1/L) = 0, 

where L = exp (-T L /T C ) and D = exp (-T/T c ).' 
If T < Tl < 27 1 , the equation becomes 

(43) z 3 - (1 + D)z 2 + [D + K{1 - D 2 /L)]z 4- KD(D/L - 1) = 0. 

Dead time does not increase the order of the characteristic equation as 
long as T L < T. When T < T L < 2T, the order of the equation is in- 
creased from 2 to 3. Further increase of the dead time (or shortening of 
the sensing time) leads to successively higher order characteristic equa- 
tions, and it can be shown that the equation becomes transcendental for 
the case of the continuous controller. It can be shown that the controlled 
variable never oscillates if all the roots lie on the positive axis of the 2-plane 
between and -f-1. This fact is used to force eq. (42) to have a double 
positive root less than. unity. The control factors leading to this aperiodic 
limit are shown in Fig. 16. It is not possible to cause the roots to vanish 
(i.e., z n = 0) unless compensation is added. Adding compensation in- 
troduces two arbitrary constants into the characteristic equation. The 
two extra constants can be used to design the system characteristic for 
vanishing roots. 
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Fig. 16. Excursion-dependent periodic control on a plant with first order time constant 
and dead time; aperiodic limit (Ref. 1). 
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Fig. 17. Excursion-dependent periodic control with rate action, on a plant with first 
order time constant and dead time; parameters for optimal response (Ref. 1). 

Second Order System with Dead Time and Rate Stabilization. 

(See Fig. 17.) This system leads to the equation 

(44) A z 2 + A x z + A 2 = 0, 

where A = 1, 

A, = K(l -D/L) + T T /T C -D/L - (1 + D), 
A 2 = D[l + K(l/L - 1) - Tj/Te-1/L], 

L = exp (-T L /T C ), 

D = exp(-T/T c ), 

M = step disturbance in m, 

F = control area. 



The control area will assume an absolute minimum if all the roots vanish. 
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This limit arises when Ai/A = and A 2 /A = 0. These conditions are 
called optimal because they lead to least control area. They are valid 
only in the range < Tl/T c < T/T c . The control area increases steadily 
with T/T c so that if one is free to choose T/T c , the most favorable operating 
conditions are obtained when T = Tl- The action following a step 
disturbance inside the control loop is shown in Fig. 18. The parameters 
for optimal response are shown in Fig. 17. 
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Fig. 18. Example of a difference equation of second order with vanishing characteristic 

values (Ref. 1). 



Third Order System with Dead Time and with Delayed Rate 
Compensation. The system of Fig. 19 leads to the following character- 
istic equation: 



(45) 



A z* + A x z 2 + A 2 z + A 3 = 0, 



where A = 1, 

Ai = K(l - D/L + pQ) - 1 - D - Q, 

A 2 = K[D/L(1 + Q) - pQ(l + D) - D - Q] + D + DQ + Q, 
A 3 = DQ[K(1 + p - 1/L) - 1], 
Q = exp (—T/Tr), and L, D, M, F are defined above (see eq. 44). 

Here again optimal response is possible with its finite control process 
similar to that shown in Fig. 18. The parameters for optimal response are 
shown in Fig. 19. 

Comparison of Continuous and Sampled-Data Controllers. If 

the process has no dead time, sampled-data control is decidedly less favor- 
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Fig. 19. Excursion-dependent periodic control with retractile followup, on a plant with 
first order time constant and dead time; parameters for optimal response (Ref. 1). 



able than the corresponding continuous control. Figure 20 shows the 
relationships which are present when there is dead time in the plant. If 
the control areas of sampled-data and continuous controllers without stabili- 
zation are compared at the aperiodic limit, the two upper solid lines of Fig. 
20 are obtained. For small values of the dead time, these two curves can 
hardly be distinguished from one another. Decidedly different relations 
are present, however, if the controller includes a stabilizing device, since 
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then a control response which terminates in a finite time can be had 
with a sampled-data controller. A comparison of control areas, Fig. 20, 
shows that sampled-data control gives appreciably better results. To be 
sure, the combination of parameters which causes vanishing roots of the 
characteristic equation is not possible for arbitrarily small dead times 
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Fig. 20. Comparison of optimal control areas, continuous versus periodic controllers, 

on a plant with first order time constant and dead time (Ref. 1). Basic plants similar 

to those of Figs. 16, 17, and 19. 



because, among other things, the gain required becomes smaller than is 
practically possible. It should be entirely acceptable to have somewhat 
overdamped response when the dead time is small. 

Summary. Aside from constructional considerations, the sampled-data 
controller is most appropriately used when the control loop has consider- 
able dead time and the controller is provided with a stabilizing device. 
Without the latter the behavior of a continuous controller is basically 
more favorable. However, with short dead times or a short sensing cycle, 
the differences between the two forms are so slight that constructional 
considerations alone can dictate the choice. It need hardly be explained 
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that the frequency and the form of the disturbance can be decisive in the 
choice of one or the other type of controller. The length of the sensing 
interval, T, is one of the most important considerations in the design of 
sampled-data systems. If possible it should be chosen equal to the dead 
time, Tl- If the specifications on the controlled response do not permit 
such a long sensing cycle, the sensing cycle must be shortened at the ex- 
pense of less damping or the dead time must somehow be reduced. If 
the dead time is very short, continuous controllers may be indicated, but 
the performance of sampled-data controllers should also be entirely ac- 
ceptable. * ' 
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Compensation, 23-01 ; see also Networks, 
d-c, compensation and Nonlinear 
systems, compensation 

a-c systems, 23-48 

backlash, 25-56, (table) 25-58 

control systems, 23-01 

d-c components, 23-18 

d-c electric networks (table), 23-29 

dead zone, 25-58 * 

feedback, 23-21 

load disturbances, 23-09 

mechanical networks, 23-42 

phase lag and lead, 23-18 

rate feedback, 23-21 

rate and lead network, 23-24 

saturation, 25-53 (table), 25-54 

typical nonlinear methods of (tables), 
25-61, 25-62 
Complement, sets, 1-03 
Complex functions, 7-02 

analytic, 7-04 

analytic continuation, 7-16 

convergence, 7-08 [7-21 

defined by linear differential equations, 

definite integrals, 7-03, 7-15 

elliptic, 7-18 

examples of, 7-03 

harmonic, 7-04 

hypergeometric equation, 7-23, 7-25 

identities, 7-04 

integral theorems, 7-05 

Laurent series, 7-08 

mapping, 7-03 

poles, 7-12 

power series, 7-08 

removable singularity, 7-12 



Complex functions, residues, 7-13 
Riemann surfaces, 7-17 
singularities, 7-12, 7-13 
zeros, 7-11 
Complex plane, 7-02 
plots, 20-54 
typical, 20-67 
Complex variables, 7-01 

formulas, 7-02 
Component equations (table), 20-02 
Components, analogous (table), 20-08 
error detectors (table), 20-18 
power amplifiers (table), 20-24 
Computer, analog, noise studies, 24-13 
autocorrelation, 24-11 
for feedback control analysis, 19-20 
Computer, digital, controller, 26-03 
storage requirements and number of 
operations, differential equations, 
ordinary (table), 14-63 
differential equations, partial (table), 

14-87 
eigenvalue problem (table), 14-48 
matrix inversion (table), 14-27 
systems of linear equations (table), 
14-27 
Conformal mappings, 10-01, (table) 10-06, 
21-12 
application to boundary value prob- 
lems, 10-09 
by elementary functions, 10-06 
equivalence, 10-04 
geometric meaning, 10-03 
linear fractional transformations, 10-05 
Schwarz-Christoffel, 10-08 
theorems, 10-02 
Continuous elements, 26-01 
Contours of constant M and a, 21-73 
Control accuracy, 19-06 
Control area, 19-06 
Control precision, 19-06 
Control ratio, 19-06 

peak, 19-16 
Control systems, see also Feedback con- 
trol; Nonlinear systems; Sampled 
data systems; Servomechanisms 
a-c systems, 20-79 
adjoint, 24-13 
analysis, classical method of, 20-28 



INDEX 



Control systems, block diagrams, 20-56; 
see also Block diagrams 
compensation, 23-01; see also Com- 
pensation 
design in the presence of noise, 24-15 
equations, examples, 20-05 
first order, response, 20-36 
fundamentals, 20-01 
open-closed loop, 23-54 
optimization, 24-18 
representation, 20-01 
response, by correlation technique, 

20-27 
response characteristics, 22-02 
second order parameters, 20-44, 20-46 

response, 20-39 
stability, see Stability 
types 0, 1, and 2, 20-67, 23-02 
log magnitude diagrams, 23-01 
representative plots, 20-67 
typical applications (table), 20-69 
Controllers, continuous and sampled-data, 
see also Sampled data system 
comparison of, 26-29 
periodic, 26-05, 26-25 
Controller synthesis, 19-13 
Convergence, power series, 7-08 
Convolution, Laplace, 9-08 
Convolution integral, application, 20-53 

approximated, 20-53 
Corner frequency, 19-06, 22-09 
Correlation, and signal structure, 17-09 
functions, see Autocorrelation; Cross- 
correlation 
Correlation technique, system response, 

20-27 
Cote's formulas for integration (table), 

14-11 
Coulomb friction, 25-33, 25-56 

simplified, 25-31 
Cramer rule, 3-09, 14-20 
Crank-Nicholson method, 14-84, 14-87 
Cross-correlation, function, 17-08, 17-24, 
24-03 
spectrum, 17-09 
Crossover frequency, phase margin at, 

approximation, 22-43 
Curve fitting, 14-06; see also Numerical 
analysis, curve* fitting 



Curve fitting, least squares, 13-14 

procedure, transient, 23-04 
Curve matching, 20-27 

Damping, absolute, 19-15 
factor, and degree of stability, 23-06 
ratio, 19-15, 20-44 

approximation, 22-41 

oscillatory transients, 20-50 
viscous, dashpot, 20-02 
Darlington, Tchebysheff-, filters, 17-32 
Dashpot, 20-02 
Data transmission, 18-01 
analog, 18-04, 18-21, 18-29 

noise and error, 18-21 
codes, see Codes 
decoder, 18-08 
digital, 18-05, 18-30 

noise and error, 18-21 
distances, 18-02 
echoes, 18-22 

and equalization, relationship, 18-24 

tolerance, 18-26 • 
encoding, 18-07 
equalization, 18-22 
error, influence of noise on, 18-21 

standards, 18-05 
facilities, 18-02 
impairment, 18-18 
modulation, 18-11 

amplitude, 18-13 

vestigial sideband, 18-14 

frequency, 18-15 

pulse code, 18-15 
multiplexing, 18-11 

auxiliary signal, 18-16 

frequency division, 18-15 

time division, 18-15 
noise, see Noise 
nominal effective band, 18-12 
power density spectrum, see Power 

density spectrum 
prediction, 18-10 
pulse shape, 18-12 
quadrature components, 18-14 

vestigial sideband transmission, 
18-28 
real time, 18-04 
ripple, 18-25 

tolerance, 18-26 
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Data transmission, signal-to-noise ratio, 
18-14 
critical, 18-21 

symbols, 18-01, 18-06 

teletypewriter, 18-17 

transfer functions, 18-23 

vestigial sideband, 18-14, 18-28 
D-c generator, transfer function, 20-14 
D-c motor, backlash, 25-57 

transfer function, 20-14 

with inertia load, 20-06 
D-c systems, compensating components, 

23-18 
Dead band, describing function, 25-22 
Dead time, 19-06, 19-14 

sampled data systems, 26-26 
Dead time and rate stabilization, sampled 

data systems, 26-28 
Dead zone compensation, 25-58 
Decibel, gain- conversion, 21-44 
Decoders, 16-07 
Decoding, data transmission, 18-07 

methods, 18-10 
Decrement factor, 19-15 
Delay time, 22-03 

describing function, 25-23 
Delta function, 8-06, 9-07 
Demodulators, modulators and, 20-80 
Descartes rule, 2-04 
Describing functions, 19-20, 25-13 

accuracy, 25-15 

amplitude sensitive nonlinearities 
(table), 25-22 

backlash (d-c shunt motor driving 
load), 25-30 
simplified (high load damping), 25-31 

compensation, 25-50 

complex nonlinearities (table), 25-30 

convergent point, 25-18 

Coulomb friction, simplified, 25-31 

dead band, 25-22, 25-24 

delay time, 25-23 

divergent point, 25-18 

frequency variant, 25-20 

granularity, 25-23, 25-28 

hysteresis, 25-22, 25-25, 25-27 

limitations, 25-15 

log-magnitude-angle plane representa- 
tion, 25-17 

method of equivalent coefficients, 25-35 



Describing functions, motor, acceleration 
limiting, 25-30 
velocity limiting, 25-30 

negative deficiency (types 1 and 2), 
25-22, 25-26 

Nichols charts, 25-51 

Nyquist diagrams, inverted, 25-16 

rapidly varying nonlinearities, 25-50 

relation to frequency response, 25-51 

relay, 25-23, 25-27 

saturation, 25-22, 25-24 

simplifying complex nonlinearities, 
25-34 

stability criteria, 25-15, 25-17 

theory, 25-14 

typical loci (table), 25-19 

variable gain, 25-23, 25-29 
Design charts, relating open loop fre- 
quency response and transient re- 
sponse, 22-18 
Design procedure, feedback control, 

19-12 
Detectors, error (table), 20-18 
Determinants, 3-08 
Difference, sets, 1-03 
Difference-differential equations, 9-20 
Difference equations, 4-01, 14-70 

analysis of sampled data systems, 
26-09 

difference operator, 4-03 

Laplace method, 4-08 

rules for particular solutions (table), 
4-06 

solution of linear, for sampled data 
systems, 26-10 
Differential equations, 5-01 

characteristic equation, 5-05, 5-13 

complex Fuchsian type, 7-23 

dependent variable missing, 5-09 

first order and first degree, 5-02 

first order but not of first degree, 5-07 

Fourier transforms, 8-16 

graphical methods, 5-15 

Heaviside operators, 8-05 

independent variable missing, 5-09 

integral transforms, 8-07 

integro, 9-18 

isoclines, method of, 5-15 

Laplace transforms, 9-1C 

linear, 5-04 
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Differential equations, numerical meth- 
ods, 5-14 
operational methods, 5-07, 8-05 
ordinary, 5-01, 14-55 
partial, 5-20; see also Numerical analy- 
sis, differential equations, partial 
Picard, method of, 5-14 
power series, 5-10 
relaxation methods, 5-22 
simultaneous, linear, 5-12 
Heaviside Operators, 8-05 
Laplace transforms, 9-14 
singular points, 5-16, 7-22 
step-by-step integration, 5-14 
successive approximations, 5-14 
Taylor series solution, 5-10 
undetermined coefficients, method of, 

5-05 
variation of parameters, 5-06 
Differential operators, 8-01 
Differentiation, numerical, 14-08 

pure, 21-31, 21-33 
Digital data, transmission, 18-05, 18-21, 

18-30 
Digital servo system, 26-20 
Dimsdale and Clippinger method, 14-60 
Dipoles, 23-15 
Dirichlet problem, 5-20, 10-10, 14-72, 

14-76 
Disturbances, see Noise 
Division, synthetic, 2-02 
Donahue's analytical procedure, 21-52, 

(tables) 21-54, 21-57 
Double description theorem, 3-16 
Duhamel theorem, 8-06 
Duty cycle, 19-18 

Dynamic error coefficients, 19-17, 20-71 
Dzung criterion, 21-10, 21-72 

Echoes, in data transmission, 18-22 
Eigenfunctions, expansion theorem, 6-07 
Eigenvalue problem, 14-28 
Eigenvalues, 3-11, 6-02 

integral equations, 6-02 

problem, variational interpretation, 
6-08 

theorems, 6-06 
Eigenvector, 3-11, 14-28 
Electrical elements (table), 20-08 

transfer functions (table), 20-14 



Electrical systems, equations, 20-02 
Element, 1-01 

Encoding, data transmission, 18-07 
Enskog, method of, 6-13 
Envelope delay distortion, 18-27 

tolerance, 18-28 
Erlang models, 15-76, 15-78 
Error, calculation, 20-74 
characteristics, type 0, 1, 2 systems 

(table), 20-69 
coefficients, 20-70, (table) 20-73 
acceleration, 20-71 
calculation, 20-74 
dynamic, 20-71, 20-74, 20-78, 23-03 
position, 20-70 
relative usefulness, 20-77 
static, 20-70, 23-01 
steady-state, 20-72 
velocity, 20-70 
correctors, (table), 20-26 
detectors (table), 20-18 
sinusoidal applied, 20-71 
Error coefficients, 19-17, 23-12 
and control system types, 23-02 
and zeros, 23-15 
criteria, 19-17 
dynamic, 23-03, 23-04 
ratio, 19-06 
static, 23-01 
Error-correcting codes, 18-11 
Error criteria, mean square, 24-15 
Error-detecting codes, 18-10 
Error functions, 7-26 
Euler constant, 7-25 
Euler-Mascheroni constant, 7-26 
Euler method, 14-57, 14-58, 14-59, 14-67 
Evans, W. R., root locus method, 21-46 
Existence theorem, 5-02 
Extraneous signals, 19-13, 23-09; see also 

Noise 
Extrapolation formulas (tables), 14-57, 
14-58 

Factor theorem, 2-02 
Farkas lemma, 3-16 
Feedback compensation, 25-62 

d-c systems, 23-21 
Feedback control, analysis (table), 19-20 
computers, 19-20 
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Feedback control, basic elements, 19-04 
block diagrams, 20-57 
common performance specifications 

(table), 19-14 
computers, 19-20 
controller, 19-06 

definitions of terms (table), 19-04 
design steps, 19-12 
optimization, 19-19 
performance specifications (table), 

19-14 
symbols (table), 19-01 
symbols alternate (table), 19-11 
synthesis methods (table), 19-19 
system, 19-06 
systems analysis, 20-01 
systems types, 20-66 
terminology (table), 19-01 
Feedback controller, 19-06 
Figures of merit, static error coefficients, 

23-02 
Filters, Butterworth-Thomson, 17-28 
transitional, 17-30 
continuous, response to sampled data, 

26-07 
correlation in a derivative, 17-20 
design data, 17-31 ' 

discrete data, 17-22 
factorization problem, 17-14 
with lag, 17-19 
minimum phase condition, 17-11 

network, 17-12, 17-14 
network synthesis, 17-25 

Cauer method, 17-26 
optimization, alternates to Wiener 

criterion, 17-24 
optimum, 17-02, 17-13, 17-19, 24-16 

decision theory, 17-24 
phase response, of shaping network, 

17-10 
power spectrum, see Power density 

spectrum 
prediction, 17-13 
example, 17-16 
nonlinear, 17-25 
time, 17-14, 17-18 
realizable network, 17-11 
symbols, 17-04 

Tchebysheff-Darlington, 17-32 
transfer response, amplitude, 17-12 



Filters, transfer response, of the optimum, 

17-13 
First order systems, Laplace transform 
pairs (table), 20-34 
responses, 20-36 
Floyd's procedure, 22-44 
Focal point, 25-38 
Fourier, coefficients, 8-11 
cosine, integral, 8-15 
series, 8-11 
transform, 8-15 
integral, 8-09, 8-15 
series, 8-10 
sine series, 8-11 

transform, 8-15 
transforms, 8-09, 8-16 
convolution, 8-12, 8-16 
exponential, 8-16 
finite, 8-10, 8-12 
inverse, 8-12 

properties, 8-16 [18-23 

Fourier transform, data transmission, 
Fredholm, integral equation, 6-01, 6-06 
Frequency domain, noise, 24-12 
Frequency modulation, 18-15 
Frequency response, 19-07, 19-20 
approximations (table), 22-41 
bandpass frequency, 22-03, 22-09 
closed loop, 22-03 
corner frequency, 22-09 
maximum, ratio of output to input, 

22-08 
maximum amplitude ratio, 22-03 
open loop, 22-03 [22-18 

and transient response, design charts, 
related parameters, 22-08 
relation to transient response, 22-01 
graphical techniques, 22-43 
numerical techniques, 22-43 
open loop design charts, 22-18 
sampled data systems, 26-17 
Samulon method, 22-48 
Shannon sampling theorem, 22-48 
transient response from, 22-44 
from transient response, 22-47 
Frequency-sensitive networks, 25-58 
Friction, simplified Coulomb, describing 

function, 25-31 
Fuchsian type differential equation, 7-23 
Functions, 1-06 
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Functions, Bessel, 7-24 
beta, 7-26 

cosine integral, 7-27 
delta unit impulse, 8-06, 9-08 
error, 7-26 

exponential integral, 7-27 
gamma, 7-25 
Green, 6-04 
Hankel, 7-24 
incomplete gamma, 7-26 
Jacobian elliptic, 7-18 
Jacobi theta, 7-20 
Legendre, 7-24 
logarithmic integral, 7-26 
Mathieu, 7-25 
propositional, 11-05 
Reimann zeta, 7-27 
scalar product of, 6-05 
sine integral, 7-27 
transcendental, 7-25 ' 

unit, 8-06, 9-06 

Weierstrass, 7-20 ■ 

Weierstrassian analytic, 7-16 
Weierstrass sigma, 7-20 
Weierstrass zeta, 7-20 
Whittaker, 7-25 

Gain, attenuation, 19-07 

constant, 21-31, 21-32 

crossover, 19-07 

-decibel conversion, 21-44, 21-45 

intermodulation effect on, 25-06 

magnitude, 19-07 

margin, 19-07, 19-16, 21-19 
Galvanometer, transfer function, 20-14 
Games, see Operations research, games 
Gamma function, 7-25 

incomplete, 7-26 
Gauss formula (table), 14-12 
Gaussian distribution, 12-16, 12-18 

entropy of, 16-43 

errors, transmission, 18-06 
Gaussian noise, 16-44, 18-18 

white, 18-19 
Gauss quadrature, 6-11 
Gears, backlash, 25-58 
Gill method, 14-59 
Graffe method, 2-06 

Gram Schmidt orthogonalization method, 
14-18, 14-27 



Granularity, describing function, 25-23, 

25-28 
Graphical solution, determination of 

poles and zeros, 23-16 
Gray code, 18-07 
Green function, 6-04 
Guillemin method, 20-79 
Gyros, rate, 23-21 
Gyroscope, transfer function, 20-15 

Hamilton-Cayley theorem, 3-11 ' 

Hankel function, 7-24 

Hartley, 16-08 

Hartley method, 14-49, 14-50 

Heat equation, 5-20, 14-70, 14-82 

Heaviside calculus, appraisal, 8-07 

expansion formula, 9-10 
theorem, 8-04 

operator, 8-01 
inverse, 8-02 
power series, 8-04 

unit function, 8-06, 9-06 
step response to, 8-06 
Hermitian matrices, 3-14, 14-38, 14-42 
Heun formula, 14-58, 14-59, 14-67 
Hilbert matrix, 14-14 
Holding circuit, 26-07 
Homogeneous equation, 20-28 
Huffman coding, 16-12 
Hurwitz criterion, 21-71 
Hurwitz-Routh criterion, 2-05 
Hydraulic elements (table), 20-08 

transfer functions (table), 20-16 
Hydraulic networks (table), 23-45 
Hysteresis, describing function, 25-22, 
25-25 

// operation, 12-01 

Impedance, complex (table), 20-55 

Implication, 11-06 

Impulse function, 9-07 

Floyd procedure, 22-44 

response to, 9-15 
Impulse response, optimum, 24-17 
Inequalities, linear, theorem, 3-17 

system of linear, 3-14 
Inertia, rotational systems, 20-02 
Information theory, 16-01 

amount of information, 16-08 

bits and binits, 16-08 
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Information theory, channels, 16-06 

binary, 16-29 

capacity of, 16-24, 16-32, 16-43 

coding theorems, 16-26, 16-34 

continuous noisy, 16-41 

discrete, 16-06 

discrete noiseless, 16-24 

discrete noisy, 16-32, 16-26 

Gaussian noise, 16-44 
codes, 16-04 

binary, 16-11 

block coding, 16-22 

codebooks, 16-04 

codewords, 16-13 

coding delay, 16-05, 16-38 

coding theorems, 16-12, 16-17, 16-26, 
16-34 

error-free coding, binary erasure 
channel, 16-35 

error probability, 16-38 

Huffman coding, 16-12 

minimax coding, 16-17 

parity check, 16-36 

pulse-code modulation, 16-40 

Shannon-Fano coding, 16-12 

transliteration, 16-05 
decoders, 16-06 

distribution of information, 16-09 
entropy, 16-11, 16-43 
messages, 16-03, 16-18 
minimax coding, 16-17 
mutual information, 16-26, 16-28, 16-42 
quantization, 16-39 
redundancy, 16-24 
sampling, 16-39 

theorem, 16-40 
segmentation, 16-03 
self-information, 16-08, 16-20, 16-21, 

16-41 
sources, 16-03 

continuous, 16-39 

controlled, 16-12, 16-16 

discrete, 16-19 

Markov, 16-20 

rate of, 16-11, 16-21 

simple discrete, 16-08 

uncontrolled, 16-16 
Szilard-Kraft inequality, 16-13 
Integral compensation, saturation, 25-54 
Integral equations, 6-01 



Integral equations, adjoint, 6-06 

approximation methods, 6-10 

boundary value problems, 6-03 

degenerate kernels, method of, 6-11 

eigenvalues, 6-02 
theorems, 6-06 

Enskog, method of, 6-13 

expansion theorem, eigenfunctions, 6-07 

kernel, 6-02 

Rayleigh-Ritz quotient, 6-09 

Ritz-Galerkin method, 6-12 

steepest descent, method of, 6-16 
Integrals, approximation of, 6-10 
Integral transforms, 8-07 

convolution, 8-08 

inverse, 8-08 
Integration, numerical, 14-11 

pure, 21-31, 21-33 
Integro-differential equations, 9-18 
Interpolation, numerical, 14-02 
Intersection, sets, 1-03 
Invariant, 1-08 

Inventory models, 15-21 ; see also Op- 
erations research, inventory models 
Isoclines, method of, 25-41 

Jacobian elliptic functions, 7-18 
Jacobi method, 14-42, 14-48 
Jacobi theta function, 7-20 
Jordan-Gauss method, 14-16, 14-27 
Jordan normal form, 3-11 
Jump resonance, 25-04 

Kirchhoff law, 20-05 

Laasonen method, 14-84, 14-87 
• Lag, see also Phase lag 

quadratic, 21-31, 21-36 

simple, 21-31, 21-34 
Lagrange formula, 14-02 
Lagrangian multipliers, 6-13, 15-12, 24-19 

modified, 15-13 
Lame equation, 7-24 
Laplace equation, 5-20, 14-70, 14-77 
Laplace method, difference equations, 

4-08 
Laplace-Stieltjes transform, 8-18 
Laplace transforms, 8-17, 9-01, (table) 
9-04 

asymptotic behavior of, 9-21 
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Laplace transforms, convolution, 9-08 
differential equation application, 9-10 
feedback control, 20-30 
inverse, 9-09, (table) 9-12, 20-31 
pairs, 20-31, (table) 20-32 
first order system (table), 20-34 
second order system (table), 20-35 
partial fraction expansion, 20-34 
sampled data systems, 26-06, 26-08, 

(table) 26-12 
two-sided, 8-18 
of unit functions, 9-06 
and ^-transforms, output, 26-15, (table) 
26-12 
Laplacian operator, 6-03 
Laurent expansion, 7-11 
Laurent series, 7-10 
Lead, see also Phase lead 
quadratic, 21-36 
simple, 21-31, 21-34 
Lead-lag network, 20-16 
Least squares, 13-14 
Lebesgue integral, 12-09 
Legendre, elliptic integrals of, 7-19 
equation, 7-23 
function, 7-24 
transform, 8-19 
Level changes, in data transmission, 
18-29 
tolerance in digital system, 18-30 
Levenberg method, 14-08 
Lewis servo, 25-62 
Liebmann extrapolated method, 14-73, 

14-87 
Limit cycles, 5-19, 25-04, 25-38 
Linear equations, systems, 14-13; see 
also Numerical analysis, linear 
equations, systems of 
Linear programming, see Operations re- 
search, linear programming 
Linear system, 19-07, 21-02 
Liouville theorem, 7-08 
Load disturbances, 23-09 
Load resonance, 25-56 
Load sensitivity, 19-15 
Log magnitude and phase diagram 

(Bode diagram), 19-07 
Log magnitude-angle charts, 23-05 ; see 

also Nichols charts 
Log magnitude-angle diagram, 19-07 



Log magnitude diagrams, 23-01 
synthesis, 23-01, 23-08 
transient curve fitting, -23-04 

Logarithmic plots, 20-56 

Loop gain, 19-07 

Loop input, 19-07 

Loop ratio, 19-08 

Mapping, see Conjormal mapping 
Markov sources, 16-20 
Mathieu equation, 7-25 
Matrices, 3-01 

adjoint, 3-09 

canonical form, 3-10, 3-12 

diagonal, 3-11 

echelon form, 3-05, 3-06 

equivalence, 3-09 

Hermitian, 3-14 

inverse, 3-07 

inversion, 14-13; see also Numerical 
analysis, matrix inversion 

Jordan normal form, 3-11 

orthogonal, 3-13 

rank, 3-07 

similarity, 3-10 

symmetric, 3-13 
Maximum operating conditions, 19-18 
Maximum principle, 7-08 
Maximum system error, 19-17 
Measurement devices, errors (table), 

20-18 
Mechanical elements (table), 20-08 

transfer functions (table), 20-13 
Mechanical-hydraulic networks (table), 

23-45, 23-48 
Mechanical networks (table), 23-42, 

23-48 
Mechanical-pneumatic networks (table), 

23-46, 23-48 
Mesh analysis, nodal and, 20-11 
Messages, 16-03 

effective number of, 16-18 
Milne procedure, 14-58 
Minimal polynomial, 11-08 
Minimax principle, 15-102 
Minimum error criterion, 19-16 
Minimum response time systems, 25-63 
Models, see Operations research 
Modulation, see Data transmission, 
modulation 
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Modulators and demodulators, 20-80 
Monte Carlo method, 13-17, 15-18, 15-97 
Morera theorem, 7-06 
Morris escalator method, 14-18, 14-27 
Motors, a-c, 23-48 

acceleration limiting, describing func- 
tion, 25-30 

backlash, 25-57 

optimum switching technique, 25-64 

split drives, 25-58 

velocity limiting, describing function, 
25-30 
Multiloop feedback systems, 23-26 
Multiloop systems, 21-28, 21-69 
Multiplexing, see Data transmission, 

multiplexing 
Murray and Brock method, 14-60 
Mutual information, 16-26, 16-42 

distribution, 16-28 

Nat, 16-08 

Natural frequency, 20-44 
damped, 22-10 
undamped, 20-44 
Natural period, 20-45 

undamped, 20-44 
Negative deficiency, describing function, 

25-22, 25-23 
Networks, a-c, bridged-T, 23-49 
carrier frequency shift, 23-52 
compensation, 23-48 
parallel-T, 23-50 
tachometers, 23-53 
d-c, compensation, 23-18 
electric (table), 23-29 
hydraulic (table), 23-45 
mechanical (table), 23-42 
minimum phase, 17-12, 17-14, 21-30 
pneumatic (table), 23-46 
shaping, 17-10 
synthesis, see Filters 
Neumann problem, 10-10, 14-72 
Neumann series, 6-15 
Neville procedure, 14-03 
Newton difference formulas, 14-08, 

(table), 14-09 
Newton interpolation formulas, 14-04 
Newton law, 20-05 
Newton method, 2-04, 15-16 
Nichols charts, 21-76, 23-05, 23-24 



Nichols charts, multiple, 21-78 

use with describing function, 25-51 
Nodal and mesh analysis, 20-11 
Nodal point, 25-38 
Noise, 17-02, 19-13, 24-01 

additive, 16-42 

autocorrelation of, 17-08 

data transmission, types, 18-18 

error, high frequency, 23-08 

error criteria, 24-15 

Gaussian, 18-18 

generators, 24-14 

impairment of margin, 18-23 

influence on error, 18-21 

measurement of, 24-06 

Rayleigh distribution, 18-20 

sampled data systems, 26-25 

system response, 24-11 
computer methods, 24-13 
frequency domain methods, 24-12 
time domain methods, 24-11 

systems design in the presense of, 24-15 

Wiener-Hoft equation, 24-18 
Nonlinearities, see also Backlash; Com- 
pensation 

amplitude sensitive (table), 25-22 

common types (table), 25-04 

complex (table), 25-30 

essential, 25-01 

to improve system response, 25-61, 
(table) 25-62 

intermodulation effect on gain, 25-06 

jump resonance, 25-04 

limit cycle, 25-04 

optimum switching functions (table), 
25-65 

parasitic, 25-02 

rapidly varying, 25-02 

simplifying complex, 25-34 

slowly varying, 25-02, 25-48 

subharmonic generation, 25-05 

typical complex (table), 25-30 
Nonlinear methods of compensation 

(table), 25-62 
Nonlinear problem, 14-84 
Nonlinear systems, 25-01 

common phenomena (table), 25-04 

compensation, 25-48, 25-61 
relay servomechanisms, 25-52 

dynamic effect, 25-02 
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Nonlinear systems, jump resonance, 25-04 
saturation effects, typical types (table), 

25-54 
synthesis, 25-03 
variable linear, 25-47 
Nonlinear systems analysis, see also De- 
scribing junctions; Phase plane 
analysis 
adjoint computing method, 25-48 
analytical solutions, 25-43 
linearization, 25-07 
graphical, 25-10 

useful algebraic approximations 
(table), 25-09 
normalized performance charts, 25-43 
perturbation theory, 25-07 
piecewise linear, 25-43 
stability, ultimate, 25-47 
Taylor series, 25-08 
typical loci for amplitude sensitive 

(table), 25-19 
variable linear, 25-47 
Nonstatoinary processes, 24-02, 24-18 
Normal distribution, 13-05 

function (table), 13-18 
Norton theorem, 20-11 
Not operation, 12-01 
Numbers, complex, 7-01 
strong law of large, 12-12 
table of random, 15-19 
weak law of large, 12-07 
Numerical analysis, curve fitting, 14-06 
Levenberg method, 14-08 
differential equation, ordinary, 14-55 
Adams and Bashforth method, 14-59, 

14-63 
Brock and Murray method, 14-60 
computer storage requirements 

(table), 14-63 
Dimsdale and Clippinger method, 

14-60, 14-63 
Euler method, 14-57, 14-58, 14-59 
extrapolation formulas (table), 14-57, 

(table) 14-58, 14-63 
extrapolation of zero grid size, 14-60, 

14-63 
fourth order method, 14-59, 14-63 
Gill method, 14-59, 14-63 
Heun formula, 14-58, 14-59, 14-63 



Numerical analysis, differential equation, 
ordinary, Milne procedure, 14-58 
number of operations (table), 14-63 
Runge-Kutta method, 14-59, 14-60, 

14-63 
Simpson rule, 14-58 
trapezoidal formula, 14-58 
differential equation, partial, 14-64 
Cauchy method, 14-66 
classification, 14-68 
Euler method, 14-67 
Heun method, 14-67 
replacement by difference equations, 
14-70 
differential equation, partial elliptic, 
14-71 
computer storage and time require- 
ments, 14-87 
Dirichlet problem, 14-72, 14-76 
iteration method, 14-72 
Laplace equation, 14-77 
Liebmann method, 14-73, 14-87 
Neumann problem, 14-72 
Peaceman and Rachford method, 

14-75, 14-87 
relaxation method, 14-72 
Richardson method, 14-73, 14-87 
differential equation, partial hyper- 
bolic, 14-77 
computer storage requirements, 

14-87 
explicit method, 14-78, 14-87 
implicit method, 14-80, 14-87 
roundoff errors, 14-80 
triple diagonal systems, 14-81 
truncation errors, 14-80, 14-87 
von Neumann criterion for conver- 
gence, 14-82 
differential equation, partial parabolic, 
14-82, 14-87 
computer storage requirements 

(table), 14-87 
Crank-Nicholson method, 14-84 
explicit method, 14-83, 14-87 
implicit equation, 14-84, 14-87 
Laasonen method, 14-84, 14-87 
truncation error, 14-87 
differentiation, 14-08 
Newton difference formulas, 14-08, 
(table) 14-09 
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Numerical analysis, differentiation, Stir- 
ling formulas, 14-09, (table) 14-10 
eigenvalue problem, 14-28 
adjoint, 14-37 

bounds on eigenvalues, 14-44 
Cayley-Hamilton theorem, 14-32 
computer storage requirements, 

14-48 
eigenvalues of special matrices, 14-43 
escalator method, 14-33 
Hermitian matrices, 14-38, 14-42 
Jacobi method, 14-42, 14-48 
number of operations for finding 

eigenvalues and eigenvectors, 14-48 
Sourian-Frame algorithm, 14-29 
Sturm sequence, 14-36 
triple diagonal method, 14-34, 14-48 
integration, 14-11 

Cote formula (table), 14-11 
Gauss formula (table), 14-12 
Simpson rule, 14-11 
trapezoidal rule, 14-11 
interpolation, 14-01 
Lagrange formula, 14-02 
Neville procedure, 14-03 
Newton formula, 14-04 
linear equations, systems of, 14-20, 
14-27 
computer storage requirements, 14-26 
conjugate gradient method, 14-21, 

14-24, 14-27 
Cramer rule, 14-20 
elimination method, 14-20, 14-27 
gradient method, 14-23, 14-27 
number of operations, 14-26 
relaxation method, 14-22, 14-27 
Seidel method, 14-21, 14-27 
matrix inversion, 14-13, 14-27 

digital computer storage require- 
ments, 14-26 
Gram Schmidt orthogonalization 

method, 14-18, 14-27 
Jordan-Gauss method, 14-16, 14-27 
modified matrix method, 14-19, 14-27 
Morris escalator method, 14-18, 14-27 
number of operations, 14-26 
partition method, 14-17 
statistical analysis of experiments, 
14-48 
balanced incomplete blocks, 14-51 



Numerical analysis, statistical analysis of 
experiments, factorial designs, 14- 
49, 14-54 
Hartley method, 14-49, (tables) 

14-50 
variance, 14-49, (tables) 14-51, 14-54 
Nyquist criterion, 21-09 
abbreviated, 21-18 
applications, 21-15 
conformal mapping, 21-12 
diagram, 19-08 
sampled data systems, 26-17, 26-19, 

26-25 
use in system compensation, 23-24, 
23-26 
disadvantage, 21-28 
multiloop systems, 21-28 
physical meaning of, 21-11 
practical considerations in plotting 

diagrams, 21-17 
principles, 21-13 
rephrased in terms of Bode diagrams, 

21-31 
use of Bode diagrams in drawing plots, 
21-43 

Open-closed loop control, 23-54 
Open loop, vs. closed loop, 19-12 
frequency response, 22-03 
poles and zeros from the closed loop, 

23-15 
response, closed loop respose from, 

21-72 
roots, relation between closed loop 

roots and, 22-15 
transfer functions, polar plots of some 
common, 21-21 
Operational mathematics, 8-01 
Operational research, see Operations re- 
search 
Operations research, 15-01, 15-04 
allocation models, 15-31 
bidding problems, 15-100 
competitive problems, 15-99 
control of the solution, 15-120 
dynamic programming, 15-31 
effectiveness, measure of, 15-08 
executive problems, 15-03 
games, 15-99 
four-by-four, 15-113 
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Operations research, games, minimax 
principle, 15-102 

mixed strategy, 15-102, 15-104 

one-person, 15-101 

rectangular games, theorems, 15-106 

single strategy, 15-102 

three-by-rc, 15-113 

three-by-three, 15-111 

two-by-n, 15-109 

two-by-two, 15-107 

two-person, zero-sum, 15-101, 15-102, 
15-104 

zero-sum, n-person, 15-115 
implementation, 15-123 
inventory models, 15-21 

dynamic, 15-30 

elementary, 15-22 

multistorage points, 15-30 

with price breaks, 15-28 

with restrictions, 15-30 
Lagrangian multipliers, 15-12 
linear programming, 1 5-3 1 

assignment problem, 15-73 

dual problem, 15-67 

games, 15-114 

geometric interpretation of, 15-63 

short cut, 15-70 

simplex method, 15-33, 15-41, 15-65 
models, 15-08, 15-10 

data reduction, 15-119 

sampling, 15-117 

solutions, 15-10 

testing, 15-115 

types, 15-10 
Monte Carlo method, 15-18, 15-96 
Newton method, 15-16 
problem formulation, 15-05 
queuing theory, 15-73 
random walk problems, 15-18 
replacement models, 15-86 

items that deteriorate, 15-86 

items that fail, 15-89 

Monte Carlo method, 15-96 
sampling, 15-117 
simplex method, 15-33 
transportation problem, 15-46 

alternate evaluation method, 15-60 

alternate optimum programs, 15-54 

northwest corner rule, 15-48 



Operations research, transportation prob- 
lem, solution of maximization 
problems, 15-57 
variations, 15-58 
waiting time models, 15-73 
Erlang, 15-76, 15-78 
holding time, 15-78 
Lindley, 15-76 
multiple channels, 15-82 
Poisson input, 15-75 
priority discipline, 15-79 
sequencing models, 15-83 
single channel, 15-77 
Operators, difference, 4-03 
Heaviside, 8-01 
integral, 8-07 
power series, 8-04 
Optimization, criterion of, 17-13 

feedback control, 19-19 
Optimum switching, functions (table), 
25-65 
techniques, 25-63, 25-64 
Or operation, 11-01, 11-03, 12-01 
Oscillation frequency, approximation, 

22-42 
Oscillations, hard and soft, 25-38 
Output transforms for basic sampled 

data systems, 26-15 
Overshoots, 19-10, 19-14 
first and successive, 23-07 

Parallel-T network, 23-50 
Parity check, 16-36, 18-10 
Peaceman and Rachford method, 14-75, 

14-87 
Peak overshoot, approximations, 22-41 
Performance index, 19-16 

sampled data systems, 26-25 
Periodic controllers, 26-01, 26-05, 26-25; 

see also Sampled data systems 
Phase angle, servomechanism scale, 21-36 
Phase crossover, 19-08 
Phase lag, compensation networks, 23-18 
electric (table), 23-34 
mechanical (table), 23-43 
pneumatic (table), 23-47 
network, 20-16 
Phase lag-lead, compensation networks, 
23-20 
electric (table), 23-38 
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Phase lag-lead, mechanical (table), 23-43, 
23-44 
pneumatic components, 23-47 
Phase lead, compensation networks, 23-18 
electric (table), 23-30 
feedback, 23-25 
mechanical (table), 23-42 
pneumatic components (table), 23-46 

network, 20-16 
Phase margin, 19-08, 19-16, 21-18 

at crossover frequency, approximation, 
22-43 
Phase plane analysis, 5-15, 25-36 

construction methods, 25-39 

isoclines, method of, 25-41 

phase trajectory, 25-36, 25-38 

relay servo, 25-40 

singular points (table), 25-38 
Phase portrait, 25-36 

significant characteristics of (table), 
25-38 
Phase space, 25-39 
Physical laws, 20-04 
Picard, method of, 5-14 
Pneumatic components (table), 23-46 
Poisson distributions, 12-16, 13-04 
Poisson equation, 14-72, 14-73 
Poisson input, 15-75 
Polar plots, inverse, 21-20 

of some common open loop transfer 
functions, 21-21 
Poles, closed loop, relation to system 
characteristics, 23-11, (table) 23-12 

complex, dominant pair, 22-04, 22-07 
Pole-zero location, synthesis, 23-11 
Polynomials, 2-01 

characteristic, 3-11 
Power amplifiers (table), 20-24 
Power amplifier saturation, 25-54 
Power density spectrum, 17-07, 24-04 

autocorrelation, 17-07, 17-14 

baseband signal, 18-11 

cross-correlation, 17-09 

phase response, shaping network, 17-10 

pulse, 18-12 

relation to autocorrelation, 24-06 

shaped impulses, 17-09 

vestigial sideband transmission, 18-14 
Power elements, selection of, 19-13 
Power series, 7-08 



Preamplifier saturation, 25-54 
Prediction, 17-03, 17-16; see also Filters 

data transmission, 18-10 

discrete data, 17-04 

symbols, 17-04 
Primary feedback ratio, 19-08 
Probability, 12-01; see also Statistics 

almost certain, 12-13 

averages, 12-07, 12-13 

Bernoulli distribution, 13-04 

binomial distribution, 12-16, 13-04 

central limit theorem, 12-13, 12-18 

conditional, 12-03 

continuous random process, 12-18 

covariance function, 12-18 

density, 12-09 

distribution functions, 12-04 

distribution of sums and averages, 
12-13 

expectation, 12-09, 13-03 

expected value, 12-06, 12-07 

independence, 12-04 

joint distribution, 12-10 

mean, 12-09, 13-03 

normal distribution, 12-16, 12-18, 13-05 

Poisson distributions, 12-16, 13-04 

postulates, 12-01 

random variables, 12-04, 13-02 
bounded, 12-08 
simple, 12-07 

sentences, 12-01 

stationary process, 12-18 

strong law of large numbers, 12-12 

theorems, 12-02 

variance, 12-11, 12-12, 13-03 

weak law of large numbers, 12-07 
Pulse code modulation, 16-40, 18-15 

Quadratic lag and lead, 21-31, 21-36 
Queuing theory, see Operations research, 
waiting time models 

Rachford, Peaceman and, method, 14-75, 

14-87 
Ramp inputs, first order system, 20-34 

second order system, 20-35 
Random inputs, see Noise 
Random processes, 24-02 

stationary, 24-04 
Rate action, controller, 26-28 
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Rate and lead network feedback, 23-24 
Rate feedback compensation, 23-21 
Rate gyros, 23-21 

inertial navigation, 23-10 
Rate network, a-c, 23-49 
Rayleigh distribution, of noise, 18-20 
Rayleigh-Ritz quotient, 6-09 
R-c networks (table), 23-29 
Real time, data transmission, 18-04 
Redundancy, 16-24 
Reflected binary code, 18-07 
Regression curve, 13-13 

function, 13-14 
Relations, 1-05 

binary, 1-05, 1-07 

equivalence, 1-07 

order, 1-09 

product of, 1-06 
Relaxation methods, 5-22, 14-22, 14-27 
Relay, describing function, 25-23 
Relay seryo, compensation, 25-51, 25-52 

phase trajectories, 25-40, 25-42 

piecewise linear, 25-44 
Remainder theorem, 2-02 
Replacement models, 15-86 
Reset, component, integrating, 23-53 

divided, 25-58 

servo, 23-54 
Residues, 7-11 
Resolution, 19-18 
Response, see Frequency response; 

Transient response 
Response time, 19-08 
Richardson method, 14-73, 14-87 
Riemann mapping theorem, 10-04 

surfaces, 7-17 

theorem of, 7-12 

zeta function, 7-27 
Ripple, 18-25 
Rise time, 19-08, 19-14, 22-03 

approximations, 22-43 
Ritz-Galerkin method, 6-12 
Rolle theorem, 2-04 
Root locus, 19-20, 21-46 

angle condition, 21-47 

asymptote, 21-51 

common plots, 21-60 

compensation, 23-27 

construction aids, 21-58 

construction theorems, 21-49 



Root locus, construction theory, 21-47 
diagram, 19-09 
Donahue procedure, 21-52, (tables) 

21-54, 21-57 
interpretation of results, 21-68 
magnitude condition, 21-47 
mechanical angle summer, 21-58 
multiloop systems, 21-69 
practical considerations in drawing, 

21-52 
procedure, 21-48 
relative merits, 21-71 
sampled data systems, 26-25 
Spirule, 21-58 
Roots, of characteristic equation, and 
stability (table), 20-53 
closed loop, iterative process for de- 
termining, 22-16 
graphical method of determination, 

22-16 
of polynomials, 2-03, 2-04 
Roots of system equations, dynamic 

error coefficients and, 20-78 
Rotational systems, equations, 20-02 
Rouche theorem, 7-15 
Routh criterion, 21-05 

special cases, 21-07 
Routh-Hurwitz criterion, 19-20 
Routh-Hurwitz stability criteria, sam- 
pled data systems, 26-16 
Runge-Kutta method, 14-59, 14-60, 14-63 

Saddle point, 25-38 
Sampled data systems, 26-01 
Bode diagrams, 26-25 ■ 

constant overshoot loci, 26-23 
control area, 26-25 
controllers, comparison of continuous 

and, 26-31 
dead time, 26-26 

and rate stabilization, 26-28 

with delayed rate, 26-29 
design procedure, 26-20 
digital computer in the controller, 26-03 
Laplace transform, 26-06, 26-08 
linearity, 26-02 

minimum control area, 26-22, 26-26 
Nyquist diagram, 26-17, 26-25 
operational instruction, 26-21 
output transforms (table), 26-15 
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Sampled data systems, performance, 
charts, 26-25 
criteria, 26-20 
index, 26-25 

regulator, 26-05 

root locus, 26-25 

smoothing, 26-07 

stability criteria, 26-15 
frequency response, 26-17 
Routh-Hurwitz, 26-16 

synthesis, 26-20 
simple analog system, 26-23 

transfer functions, 26-08 

variables between sensing instants, . 
26-22 

z-transform method, see z-transforms 
Samples, 13-02, 13-06, 26-01 

analysis of, 26-06 

variance, 13-07 
Sampling, 16-39 

Shannon theorem, 16-40, 24-07 
Samulon method, 22-48 

error, 22-49 
Saturation, compensation, 25-53, (table) 
25-54 

describing function, 25-22 

in feedback, 25-54 

power amplifier, 25-54 

preamplifier, 25-54 

servomechanisms, relay, 25-51 

typical types of (table), 25-54 
Schedule and trim control, 23-54 
Schwarz-Christoffel mappings, 10-08 
Schwarz constants, 6-15 

inequality, 6-05 
Second order systems, equation coef- 
ficients (table), 20-46 ■ 

parameters (table), 20-44 

time responses (table), 20-35, 20-39 
Segmentation, 16-03 
Seidel method, 14-21, 14-27 
Self-information, of a symbol, 16-21 , 

of continuous signals, 16-41 

Markov process, 16-20 
Selsyns, 23-48 
Sentences, algebra of, 11-05 

probability, 12-01 
Series, Fourier, 8-10 

Laurent, 7-10 

Neumann, 6-15 



Series, power, 7-08 

Taylor, 7-09 
Series networks, a-c compensation, 23-48 
Servomechanisms, 19-09 

a-c systems, 20-79, 23-48 

error coefficients (table), 20-73 

error correctors (table), -20-26 

error detectors (table), 20-18 

power amplifiers (table), 20-24 

system type, 20-67; see also Control 
system, types ...... 

Sets, 1-01 

binary operations, 1-08 

binary relations on, 1-05 

Cartesian product, 1-04 

complement, 1-03 

difference, 1-03 

empty, 1-03 

examples of, 1-01 

intersection, 1-03 

lattice, 1-09 

of points, 1-02, 1-10 

power set, 1-03 

subsets, 1-02 

union, 1-03 
Settling time, 19-09, 19-14, 22-03 

approximations, 22-41 
Shannon-Fano coding, 16-12 
Shannon sampling theorem, 22-48, 24-07 
Sheffer stroke operation, 11-02, 11-10 
Signals, extraneous, see Noise 
Signal flow diagram, 20-57 
Signal-to-noise ratio, data transmission, 

18-14 
Simplex technique, linear programming, 
15-33 

minimization problems, 15-45 
Simpson rule, 6-11, 14-11, 14-58 
sin x/x values (table), 22-52 
Singular points, 5-16, 7-22 
Singularities, 7-11 
Sinusoidal driving function, 20-54 
Smoothing, 17-01 ; see also Filters 

and decoding, data transmission, 18-08 

noise, 17-02 , 

symbols, 17-04 

Wiener theory, 17-02, 17-13 
Sources of information, see Information 

theory, sources 
Souriau-Frame algorithm, 14-29 
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Spectrum, autocorrelation and cross-cor- 
relation, see Power density spec- 
trum 
Spirule, 21-58, 22-16 
Spring loaded gears, 25-58 
Stability, 19-09, 21-01 

absolute, 20-29 

Bode diagrams, see Bode diagrams 

boundary theory, 21-72 

classical approach, 21-02, 21-04 

conditional, 21-19 

contours of constant M and a, 21-73 

criteria, z-transforms, 26-15 

degree of, 23-06 

describing function, 25-15 

Dzung criterion, 21-10, 21-72 

gain margin, 21-19 

Hurwitz criterion, 21-71 

inverse polar plots, 21-20 

margin, 21-72 

multiloop systems, 21-28 

Nichols charts, 21-76 

Nyquist criterion, see Nyquist criterion 

phase margin, 21-18 

relation to characteristic equation, 
21-03 

relation to roots of characteristic equa- 
tion (table), 20-53 

root locus method, see Root locus 

Routh criterion, see Routh criterion 

sampled data systems, 26-15 

unconditional, 21-19 

unstable system, 21-01 

Wall criterion, 21-72 
Stabilization, by internal feedback, 23-16 
Stabilization networks, for d-c compensa- 
tion (table), 23-29 
Stabilizing networks, transfer functions 

(table), 20-15 
Standard deviation, 12-12, 13-03 
Static accuracy, 20-70 
Static error coefficient, 19-17, 20-70 
Stationary processes, 24-02, 24-15 
Statistics, 13-01 ; see also Probability 

analysis of experiments, see Numeri- 
cal analysis, statistical analysis of 
experiments 

Bernoulli distribution, 13-04 

binomial distribution, 13-04 

bivariate distributions, 13-13 



Statistics, computation, 13-08, (table) 
13-09 

confidence intervals, 13-10, 13-12, 13-15 

curve fitting by least square, 13-14 

distribution of sample moments, 13-09 

expectation, 13-03 

goodness of fit, 13-16 

hypothesis testing, 13-10 

maximum likelihood, method of, 13-16 

mean, 13-03 

mean deviation from the mean, 13-04 

median, 13-04 

midrange, 13-04 

mode, 13-04 

moments, 13-03 

Monte Carlo method, 13-17 

normal distribution, 12-16, 13-05, 
(table) 13-18 

Poisson distribution, 13-04 

probable error, 13-04 

random variables, 13-02 

regression curve, 13-13 

relation to probability, 13-01 

sample space, 13-02 

sequential analysis, 13-16 

standard deviation, 13-03 

student t distribution, 13-06, (table) 
13-20 

unbiased estimate, 13-07 

variance, 13-03, 13-07 

X2 distribution, 13-05, (table) 13-19 
Steady state, data for evaluating transfer 
functions, 20-23 

equation terms (table), 20-55 

errors (table), 20-71, 20-72 

quasi-static assumption, 20-23 

response, 20-01, 20-55 

solution, 20-54 
Steady-state condition, 21-11 
Steady-state error, 19-09, 19-14 

coefficient, 19-17 
Step function, 9-07 

design charts, comparison of frequency 
response and transient response, 
22-21 

time responses (table), 20-48 
Step input, first order system, 20-34 

second order system, 20-35 
Stieltjes integral, 12-09 
Stirling formula, 14-09, (table) 14-10 
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Stirling numbers of the second kind, 4-07 
Stone, representation of Boolean algebra, 
11-9 

theorem, 11-10 
Student t distribution, 13-06 
Sturm sequence, 14-36 
Sturm theorem, 2-04 
Subharmonic generation, 25-05 
Summing points, 20-61 
Superposition, 20-10, 21-02 

principle, 5-06, 8-06 
Suppressed carrier system, 20-82 
Symbols, alternate, 19-11 

feedback control, 19-01 

smoothing and filtering, 17-04 
Synthesis, 20-03 

controller, 19-13 

dominant pair of complex conjugate 
poles, 23-11 

Guillemin method, 20-79 

log magnitude diagrams, 23-01, 23-08 

network, see Filters 

Nyquist diagram, 23-27 

pole-zero location technique, 23-11 

root locus, 23-28 

selection of methods, 19-19 
System analysis, see Control systems 
System characteristics, additional poles 
and zeros, 23-13 

relation to closed loop poles, 23-12 

two complex conjugate poles, 23-12 
System response to noise, 24-11 
System stability, see Stability 
Systems, physical (table), 20-02 

physical laws, 20-04 
Szilard-Kraft inequality, 16-13 

Tachometers, 23-21, 23-25 

a-c, 23-48 
Tandem compensation, 25-62 
Taylor series, 7-09 

differential equation solution, 5-10 
Tchebysheff-Darlington, filters, 17-32 
Tchebysheff inequality, 12-12 
Temple quotients, 6-15 
Terminology, feedback control, 19-01 
Thevenin theorem, 20-10 
Thomson, Butterworth-, filters, 17-28 
Time constant, 19-09 

equivalent, approximation, 22-42 



Time constant, for second order system, 

20-45 
Time delay, pure, 21-39 
Time domain, noise, 24-11 
Time to first peak, 22-02, 22-06 
Time parameter ratio, 20-45 
Time to peak, 22-10 

approximation, 22-41 
Time response, approximate, 22-04 

approximations, 22-41 

convolution integral, 20-53 

first order system, 20-36 

second order system, 20-35, 20-39 

transient modes (table), 20-48 
Time sequence, quasi-stationary, 17-02 

stationary, 17-02 
Time series, nonstationary, optimum 

filter design, 17-24 
Total overshoot, 19-10 
Transfer function, 9-10, 9-19, 19-10, 
(tables) 20-12, 20-13, 20-14, 20-16 

in data transmission, 18-23 

d-c motor, 25-57 

electrical elements (table), 20-14 

experimental evaluation, 20-23 

hydraulic elements, 20-16 

mechanical elements (table), 20-13 

polar plots of some common open 
loop, 21-21 

root locus plots of common, 21-60 

sampled data systems, 26-08 

typical, 20-12 
Transfer locus, 19-10 
Transfer response, amplitude," 17-12 

filters, 17-26 

of the optimum filter, 17-13 
Transformations, inverse, 3-07 

linear, 3-03 

rank, 3-03, 3-07 
Transformers, stabilizing, 23-21 
Transient error, 19-10 
Transient overshoot, 19-10, 19-14 
Transient response, 19-08, 20-01 

approximations (table), 22-41 

coefficients of terms, 22-12 

data for evaluating transfer functions, 
20-23 

delay time, 22-03 

effect of poles and zeros, 22-09 

effect of significant real poles, 22-11 
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Transient response, Floyd's procedure, 
22-44 
frequency response from, 22-47 
from frequency response, 22-44 
and open loop frequency response, de- 
sign charts, 22-18 
parameters, 22-02 

relation to frequency response, 22-01 
graphical techniques, 22-43 
numerical techniques, 22-43 
open loop design charts, 22-18 
rise time, 22-03 
settling time, 22-03, 22-10 
sin x/x (table), 22-52 s> 

•time to the first peak, 22-02, 22-06, 

22-10 
unit step input, 22-09 
Transients, input signals, 23-04 
oscillatory, 20-50 
time responses (table), 20-48 
Translation systems, equations, 20-02, 

(table) 20-08 
Transmission, of data, see Data trans- 
mission 
Transportation problem, see Operations 

research, transportation problem 
Trapezoidal formula, 14-11, 14-58 
Trapezoidal rule, 6-11 

Ultimately controlled variable, 19-10 , 

Union, sets, 1-03 

Unit function, 8-06, 9-06 

Unit impulse, first order system, 20-34 

second order system, 20-35 
Unit impulse function, 8-06, 9-08 

response to, 9-20 
Unstable system, 21-01 . 

Valve-pistons, transfer function, 20-16 
Variable gain, describing function, 25-23 
Variance, 12-11, 13-03 



Variance, analysis of, 14-49, 14-54 
Vector, components, 3-04 

convexity, 3-15 

coordinates, 3-04 

half-space, 3-14 

space, 3-01 

subspace, 3-02 
Vibrating string, 5-21, 14-70, 14-77 
Volterra integral equation, 6-02, 6-15 
von Neumann criterion for convergence, 
14-82 

Waiting time models, see Operations re- 
search, waiting time models 
Wall criterion, 21-72 
Wave equation, 5-20, 6-03 
Weierstrass, and Casorati, theorem of, 
7-12 

(P-function, 7-20 

sigma function, 7-20 

zeta function, 7-20 
Weierstrassian analytic function, 7-16 
Weighting function, 9-19 
Whittaker function, 7-25 
Wiener-Hoft equation, 24-18 
Wiener theory of filtering, 17-02, 17-13 
Wronskian determinant, 5-07 
Wye-delta transformation, 20-10 

Zeros, 7-11 

effect on bandwidth, 23-15 

effect on error coefficients, 23-15 
2-transforms, 26-11, (table) 26-12 

block diagram algebra, 26-13 

closed loop, 26-24 

design procedure, 26-20 

hidden oscillations, 26-11 

inverse, 26-13 

Laplace and (table), 26-12 

output (table), 26-15 

stability criteria, 26-15 



